| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-18 | |||
| 16:30:29 | efried | that was in fact the bug I was trying to express (by writing a test for it) when I discovered the previous. | |
| 16:30:54 | mriedem | dansmith: so no major issues with you for this unnumbered resource limitation thing? | |
| 16:31:06 | mriedem | as noted, it's no worse than what we have today | |
| 16:31:13 | dansmith | mriedem: no, I didn't have it in my head, but Idon't think it changes anything | |
| 16:31:13 | dansmith | right | |
| 16:31:21 | cfriesen | mriedem: efried: why would the compute node provider report the shared storage amounts? shouldn't it just report that it has access to a particular shared storage? | |
| 16:31:37 | dansmith | I don't like the asymmetry, but I think we probably need to keep it to avoid breakage in the meantime | |
| 16:31:38 | efried | cfriesen Nono, the compute node *has* local storage. | |
| 16:31:49 | dansmith | it's really the resourcesN being different that I hadn't grokked | |
| 16:32:28 | efried | cfriesen So it's got a local disk, and it's also attached to a SAN or whatever. The former is reported in the compute node RP; the latter via the shared RP. | |
| 16:32:41 | dansmith | what efried said | |
| 16:32:51 | dansmith | that's not possible today, but should be eventually | |
| 16:33:17 | cfriesen | efired: yeah, that makes sense. mriedem's comment made it sound like the compute node RP was reporting on sizes of available shared storage | |
| 16:34:21 | mriedem | don't compute nodes that are getting storage from NFS today report disk_gb for the entire NFS cluster? | |
| 16:34:33 | cfriesen | mreidem: yeah, and that's been a bug for a long time I think. | |
| 16:34:44 | mriedem | so it makes it look like you have potentially 100 computes with 1TB of storage each, but not really | |
| 16:34:56 | efried | https://bugs.launchpad.net/nova/+bug/1724613 < there's the first one. | |
| 16:34:57 | openstack | Launchpad bug 1724613 in OpenStack Compute (nova) "AllocationCandidates.get_by_filters ignores shared RPs when the RC exists in both places" [Undecided,New] | |
| 16:35:26 | cfriesen | incidentally, who updates the stats for the shared storage RP? is there an auditor somewhere? | |
| 16:35:32 | mriedem | yeah so i wasn't sure how we are goign to fix the NFS disk reporting thing with shared providers, because the compute service is reporting that disk_gb | |
| 16:35:41 | mriedem | cfriesen: was supposed to be external to nova | |
| 16:35:47 | mriedem | like how neutron reports IP allocation pools | |
| 16:36:05 | efried | mriedem Multiple compute services report the same inventory, but to the same RP, because the shared RP has a UUID that they can all agree on. | |
| 16:36:25 | efried | I think that's what jaypipes update_inventory_if_needed thingy is for (or whatever it's called) | |
| 16:36:27 | mriedem | efried: i don't think that's quite accurate | |
| 16:36:48 | mriedem | each compute node get_inventory is going to report what it thinks it's local disk is, | |
| 16:36:52 | mriedem | but it can't tell if it's shared or not | |
| 16:37:08 | efried | We talking Q or later? | |
| 16:37:14 | mriedem | this is why we have the 'is_shared_storage' ssh stuff during migration | |
| 16:37:33 | mriedem | well, shared storage is not Q, so later, | |
| 16:37:39 | mriedem | but this is why it's not Q, among other reasons | |
| 16:38:11 | efried | Yeah, so each compute node happily reports all the storage, but the conductor (or whatever is doing the rollup) can see that those inventories are coming from the same RP, so it can report the total just once rather than adding it up. | |
| 16:38:31 | dansmith | efried: mriedem right, computes will need to not report shared storage | |
| 16:38:33 | efried | But wait, the compute node shouldn't be reporting inventory in the compute node RP for storage that's in an aggregate. | |
| 16:38:34 | dansmith | not just report the same | |
| 16:38:44 | efried | yeah, that % | |
| 16:38:45 | dansmith | computes will report their storage if they have some, else none | |
| 16:38:45 | openstackgerrit | Merged openstack/nova-specs master: Granular Resource Request Syntax https://review.openstack.org/510244 | |
| 16:38:48 | mriedem | efried: the omputes aren't aware of hte aggregate | |
| 16:39:32 | efried | dansmith So who's responsible for creating the shared storage RP and its inventory? | |
| 16:39:39 | dansmith | efried: some storage agent | |
| 16:39:42 | dansmith | like neutron does | |
| 16:39:46 | mriedem | thinking back on pike issues, this was also a thing where the scheduler would claim disk_gb on the shared storage RP, but the compute would overwrite the instance disk_gb allocation against it's local compute node | |
| 16:39:51 | cdent | mriedem: the rt has an aggregate map (as yet unused) | |
| 16:39:51 | mriedem | because it wasn't aware of the aggregate relationship for disk | |
| 16:40:00 | cdent | that was supposed to allow it to be able to report inventory correctly | |
| 16:40:06 | cdent | once shared exists | |
| 16:40:13 | mriedem | cdent: ah, fun | |
| 16:40:29 | cdent | “fun” | |
| 16:40:46 | mriedem | i remember working a patch for one afternoon late in pike rc time trying to sort out how to not get the rt to overwrite the shared disk allocation and it went down the hole fast | |
| 16:41:10 | mriedem | involved basically reverse engineering the logic in placement and the scheduler, from the rt | |
| 16:41:25 | cdent | wheeee! | |
| 16:41:34 | mriedem | but, another reason why we don't want the RT trying to figure out allocations | |
| 16:41:43 | cdent | the aggregate_map still leaves open the question of whether a compute node can have both local and shared, which, unsure | |
| 16:42:27 | mriedem | yeah idk, couldn't you mount an NFS share on a compute and configure the instance path to use that share, but leave root and everything else that's local disk for the OS and running nova-compute? | |
| 16:42:42 | mriedem | like, i want all my instance and image crap to go in the NFS share | |
| 16:42:52 | mriedem | leave local disk for everything else | |
| 16:43:03 | dansmith | there's lots of things that need to change on compute to make that duality possible | |
| 16:43:07 | dansmith | the easiest to do today would be local disk + ceph I think, | |
| 16:43:17 | dansmith | since it's not fighting over /var/lib/instances | |
| 16:44:04 | mriedem | ok well this is why no shared storage support in queens :) | |
| 16:44:08 | dansmith | cha | |
| 16:44:12 | mriedem | granular request syntax spec approved | |
| 16:44:19 | cdent | huzzah | |
| 16:44:21 | dansmith | efried: get to work | |
| 16:45:03 | efried | Thanks for that. (jaypipes should really read it at some point, since it was his idea.) | |
| 16:45:14 | mriedem | artom: you want to start working on squashing https://review.openstack.org/#/c/512406/ into https://review.openstack.org/#/c/471356/ ? | |
| 16:50:37 | jmccarthy | Trying to figure out about using horizon to view instance console (instance on xen compute) - the console.log works, but not the console, any ideas ? | |
| 16:52:35 | artom | mriedem, sure | |
| 16:52:48 | artom | I wonder what'll come first - CI allowed you to merge it, or the next solar eclipse | |
| 16:54:30 | mriedem | or me going to lunch | |
| 16:55:45 | artom | Maybe you could eat the sun, two birds with one stone | |
| 16:56:28 | mriedem | i'm gonna need some psilocybin to pull that off | |
| 16:56:48 | openstackgerrit | sean mooney proposed openstack/nova-specs master: Use neutron's new port binding API https://review.openstack.org/375580 | |
| 16:57:45 | artom | That was awfully erudite way of asking for shrooms | |
| 17:00:25 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/newton: Use VIR_DOMAIN_BLOCK_REBASE_COPY_DEV when rebasing https://review.openstack.org/471356 | |
| 17:00:45 | artom | mriedem, ^^ | |
| 17:13:34 | openstackgerrit | John Garbutt proposed openstack/nova master: Keep updating allocations for Ironic https://review.openstack.org/513085 | |
| 17:21:45 | jmccarthy | Any ideas about how the console connection is setup to it can work via horizon ? (with instances on xen) | |
| 17:23:14 | sean-k-mooney | stephenfin: just looked at https://review.openstack.org/#/c/361140/30. there are some minor inaccuacyies in the spec but nothing i care about enough to warrent updating now that its merged | |
| 17:26:31 | sean-k-mooney | jmccarthy: i belive with xen you can still use spice/vnc or xens own console | |
| 17:26:48 | efried | cdent https://bugs.launchpad.net/nova/+bug/1724633 | |
| 17:26:49 | openstack | Launchpad bug 1724633 in OpenStack Compute (nova) "AllocationCandidates.get_by_filters hits incorrectly when traits are split across the main RP and aggregates" [Undecided,New] | |
| 17:27:44 | jmccarthy | sean-k-mooney: True, those do work (at least the xen console that I tried earlier). I was hoping to see about getting it going via horizon | |
| 17:28:28 | sean-k-mooney | jmccarthy: connect it to horizon should work all you need to do is deploy the same proxy service on dom0 that you would use for kvm | |
| 17:29:17 | sean-k-mooney | horrizon bassically uses a html5 canvas and a websocket to deliver the vnc/spice console | |
| 17:29:51 | melwitt | looks like zuul was restarted again, so check to see if you need to recheck your patches | |
| 17:30:44 | jmccarthy | sean-k-mooney: Not sure I follow here about proxy service that is needed on xen compute ? | |
| 17:31:28 | cdent | thanks efried will pay real attention a bit later | |
| 17:31:42 | sean-k-mooney | jmccarthy: actully it does not need to be on the computes. for kvm we deploy nova_novncproxy on the contoler and it connect to the hypervisor vnc console and proxyies it to horizon via a websocket | |
| 17:31:48 | efried | cdent The bug doesn't have anything you don't already know. I'll post the reviews in a bit and poke ya. | |
| 17:32:04 | cdent | s’cool, good to have on radar | |
| 17:32:56 | jmccarthy | sean-k-mooney: Ah ok yep I have that - I have both types of computes in this setup actually | |
| 17:34:07 | sean-k-mooney | jmccarthy: ya so backically in the nova compute nodes you have to add eitehr a vnc or spice section to the nova.conf or nova-compute.conf that points the hypervior at the proxy | |
| 17:34:44 | sean-k-mooney | jmccarthy: somthing like this http://paste.openstack.org/show/624007/ | |
| 17:34:45 | mriedem | need another specs core on this one https://review.openstack.org/#/c/509326/ | |
| 17:35:04 | jmccarthy | sean-k-mooney: I think I have that tho ? The console in horizon is working for instances on kvm compute | |
| 17:35:46 | sean-k-mooney | jmccarthy: is xen configured to use a xen console for guests or a vnc console. you will have to match them up | |
| 17:38:07 | sean-k-mooney | jmccarthy: are you using xen via libvirt | |
| 17:38:37 | jmccarthy | sean-k-mooney: yep | |