| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-07 | |||
| 17:19:02 | cdent | well that's not confusing at all | |
| 17:19:04 | idlemind | sean-k-mooney and bkero- http://paste.openstack.org/show/693686/ | |
| 17:19:42 | idlemind | w/o that feature policy='require' name='svm' i don't get svm in my guest and kvm-ok fails | |
| 17:20:00 | mriedem | cdent: you weren't in the room on friday | |
| 17:20:05 | mriedem | when my head exploded | |
| 17:20:25 | cdent | that was the Nth time where N is ? | |
| 17:20:27 | sean-k-mooney | idlemind: and if you do a cat /proc/cpuinfo on the host you see svm listed | |
| 17:20:48 | hrw | stephenfin: thanks for +2 ;) now have to fetch another one and +w ;D | |
| 17:20:49 | idlemind | sean-k-mooney yes w/the feature line i get svm in /proc/cpuinfo ... w/o the feature line no dice | |
| 17:21:08 | sean-k-mooney | idlemind: no i mean on the host not the guest | |
| 17:21:24 | mriedem | bauzas: dansmith: given the rebuild hoopla, and cve regression circus, we should probably get this in and backport as well https://review.openstack.org/#/c/546268/ | |
| 17:21:42 | mriedem | while i'm thinking about rebuild | |
| 17:21:55 | idlemind | sean-k-mooney yes for all 4 cores | |
| 17:21:57 | mriedem | hrw: i've got a tab open, will look in detail later | |
| 17:22:01 | hrw | mriedem: thanks | |
| 17:22:41 | sean-k-mooney | idlemind: strange so ya i think this is a libvirt/kvm/qemu bug and it it is not correctly reading the cpu flags | |
| 17:23:23 | sean-k-mooney | idlemind: i assume you are using qemu/libvirt for the rdo repos not the default centos one? | |
| 17:24:17 | idlemind | sean-k-mooney i'm not sure tbh i'm thinking they would have been brought in by openstack-ansible so i'd have to dig around there | |
| 17:24:52 | idlemind | http://paste.openstack.org/show/693687/ | |
| 17:25:20 | idlemind | looks like a fairly new release | |
| 17:25:23 | sean-k-mooney | the -ev rpms are form rdo | |
| 17:25:42 | sean-k-mooney | the default centos ones are old and busted in many ways | |
| 17:26:46 | dansmith | mriedem: I don't understand the logic in the comment you're removing | |
| 17:26:50 | openstackgerrit | Merged openstack/nova master: Fix version cap when no nova-compute started https://review.openstack.org/549737 | |
| 17:27:06 | dansmith | mriedem: seems like cases 2 and 3 are both covered by recreate=True, why was it even considering the scheduled_node in there? | |
| 17:28:38 | idlemind | sean-k-mooney well at least that's a good sign! | |
| 17:28:43 | mriedem | dansmith: honestly i think the comment was just overthinking things when it was written to fix a similar type of issue with forced host evacuate | |
| 17:28:59 | mriedem | that's why i removed it and just changed the code to check the recreate flag | |
| 17:29:31 | mriedem | the scheduled_node thing was tied into whether or not you were doing a forced host evacuate, | |
| 17:29:43 | mriedem | and the comment was added at the same time as fixing a bug in that scenario, so it was probably just 'on the brain' | |
| 17:30:21 | mriedem | that comment also predates the changes to run the host back through the scheduler on rebuild if the image changes | |
| 17:30:39 | dansmith | mriedem: yeah I just want to make sure I understand what it was trying to do | |
| 17:30:49 | mriedem | anyway, all related to said head explosion at the ptg | |
| 17:30:58 | dansmith | mriedem: looking at the original patch, "if scheduled_node is not None" which is was already there, | |
| 17:31:04 | dansmith | so it was probably just trying to augment that case, | |
| 17:31:14 | dansmith | when really it should have just considered the recreate flag itself | |
| 17:31:46 | mriedem | i'd have to go back and look at that patch to get it all back into context | |
| 17:32:10 | mriedem | i think it boiled down to, forcing a host on evacuate never worked since it was added in newton until that patch fixed it :) | |
| 17:32:30 | mriedem | was also somehow related to pci devices, so maybe it was that it never claimed? can't remember | |
| 17:32:46 | mriedem | the claim is what messes with the pci devices in the migration context right? | |
| 17:32:53 | dansmith | idk | |
| 17:32:57 | mriedem | heh same here | |
| 17:38:17 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Only attempt a rebuild claim for an evacuation to a new host https://review.openstack.org/550545 | |
| 17:41:33 | sean-k-mooney | mriedem: there are two things that mess with pci device for migration. one we dont update allcoation correctly and 2 we dont regenerate the xml for that pci passhtrough section. | |
| 17:42:14 | sean-k-mooney | mriedem: that not a new thing though livemigration with pci devices never worked | |
| 17:42:45 | mriedem | artom is going to fix it | |
| 17:42:49 | mriedem | or die trying | |
| 17:43:09 | mriedem | sean-k-mooney: what do you mean by "don't update allocation correctly"? | |
| 17:43:12 | mriedem | claims? | |
| 17:43:22 | sean-k-mooney | mriedem: yes | |
| 17:43:23 | mriedem | or something in the pci tracker? | |
| 17:43:26 | mriedem | that's all voodoo to me | |
| 17:43:34 | sean-k-mooney | well the pci device table in the db | |
| 17:43:53 | sean-k-mooney | when we migrate we dont claim the device on the new node if i remeber correctly | |
| 17:44:10 | sean-k-mooney | and i dont think we release it on the old node either | |
| 17:45:34 | sean-k-mooney | if someone does fix that then awsome but i basically equate pci passthough = no migration even though we can make it work if we really want | |
| 17:46:26 | mriedem | maybe i'm thinking about pinned cpus | |
| 17:46:28 | mriedem | L226 https://etherpad.openstack.org/p/nova-ptg-rocky-placement | |
| 17:47:05 | sean-k-mooney | pinned cpus has a similar issue in that we dont regenerate the xml to account for the available cpus on the destination | |
| 17:47:29 | mriedem | right i think that's what artom is going to spec up and fix | |
| 17:47:44 | sean-k-mooney | mriedem: artom or stephenfin ? | |
| 17:47:47 | mriedem | artom | |
| 17:48:25 | mriedem | i guess live migrating instances with pci devices will wait for nested RPs (or something) | |
| 17:48:37 | sean-k-mooney | that would be nice to fix too. i think cold migrate works today but just not live migrate | |
| 17:49:18 | sean-k-mooney | mriedem: ya or after the multiple port binding work is done perhaps. | |
| 17:49:49 | sean-k-mooney | i think cold migrate might work with pci passhtouhg though not sure. | |
| 17:49:51 | mriedem | the cpu pinning spec is going to follow the port binding template | |
| 17:50:32 | sean-k-mooney | mriedem: for the fallback and version check in the conductor to ensure both nodes support it | |
| 17:50:45 | mriedem | yes something like that | |
| 17:50:55 | mriedem | or that the requested cpus to pin are available on the dest | |
| 17:50:58 | mriedem | and/or | |
| 17:51:05 | mriedem | if not, fail that host and try another one | |
| 17:51:51 | sean-k-mooney | ya that makes sense however with the other placement discusstion about modeling dedicated cpus as a new inventory placement can gurantee that for us | |
| 17:52:15 | mriedem | is there anything placement *can't* do? | |
| 17:52:29 | mriedem | i wish placement would have my babies | |
| 17:52:30 | sean-k-mooney | well we will still need the numa toplology filter i guest but we should be able to confirm the cpus are free before calling the compute node | |
| 17:53:39 | openstackgerrit | Merged openstack/nova master: Handle not found error on taking snapshot https://review.openstack.org/511074 | |
| 18:06:10 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: Snapshot https://review.openstack.org/543023 | |
| 18:12:38 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Only attempt a rebuild claim for an evacuation to a new host https://review.openstack.org/550555 | |
| 18:22:52 | cdent | I'm going to remember this one for later [t 1nc3] | |
| 18:22:52 | purplerbot | <mriedem> i wish placement would have my babies [2018-03-07 17:52:29.605606] [n 1nc3] | |
| 18:25:30 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Fix version cap when no nova-compute started https://review.openstack.org/550557 | |
| 18:29:42 | dansmith | mriedem: do you know what this means? | |
| 18:29:42 | dansmith | http://logs.openstack.org/94/550194/4/check/nova-next/67f6ca9/job-output.txt.gz#_2018-03-07_18_22_35_715619 | |
| 18:30:33 | mriedem | bash something or other? | |
| 18:30:35 | mriedem | no | |
| 18:31:04 | mriedem | my guess would be related to something with source ${BASE}/new/devstack/lib/nova | |
| 18:31:43 | mriedem | that nova script will source the common function stuff and since we don't have stackrc sourced, GITDIR isn't defined? | |
| 18:32:05 | dansmith | yeah I bet I need to source that before nova, | |
| 18:32:13 | dansmith | else it uses gitdir as not associative | |
| 18:32:18 | dansmith | or something | |
| 18:32:24 | dansmith | however, it's doing the thing we want now | |
| 18:32:56 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Only attempt a rebuild claim for an evacuation to a new host https://review.openstack.org/550560 | |
| 18:32:56 | mriedem | it doesn't make the script fail does it? | |
| 18:33:26 | dansmith | mriedem: well, it's failing so I dunno what else caused it | |
| 18:33:38 | mriedem | i mean, if it's just noise, meh? we could set it to GITDIR=$BASE/nova if we cared | |
| 18:33:56 | dansmith | "*** FAILED with status: 1" | |
| 18:34:00 | dansmith | job failed | |
| 18:34:22 | mriedem | yeah http://logs.openstack.org/94/550194/4/check/nova-next/67f6ca9/logs/devstack-gate-post_test_hook.txt.gz#_2018-03-07_18_22_35_714 | |
| 18:35:15 | mriedem | this is nice v | |