| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-28 | |||
| 15:04:17 | sean-k-mooney | its annoying but thats all | |
| 15:05:06 | mriedem | artom: yes https://review.openstack.org/#/q/I217fba9138132b107e9d62895d699d238392e761 | |
| 15:05:29 | mriedem | me thinks we should probably start an rc-potential etherpad... | |
| 15:05:34 | sean-k-mooney | cfriesen: do you have time to repsin that by the way? if not ill add https://review.openstack.org/#/c/635350 to my list and ill file a bug | |
| 15:05:38 | mriedem | because there is just too much crap flying around for me to keep in my brain | |
| 15:06:51 | artom | mriedem, you mean rc blocker? | |
| 15:08:05 | mriedem | https://etherpad.openstack.org/p/nova-stein-rc-potential | |
| 15:08:08 | mriedem | melwitt: ^ | |
| 15:08:09 | sean-k-mooney | artom: not so much blocker but might warrent a second rc or shoudl be merged after ff as part of an rc | |
| 15:09:01 | artom | mriedem, so, sorry to pester, but the func tests blocker thing wasn't resolved | |
| 15:09:59 | mriedem | you want closure from me in other words? | |
| 15:10:26 | artom | mriedem, heh, yeah. A sense of priorities, o wise leader :) | |
| 15:12:44 | mriedem | well i personally think it's pretty risky to land something as complicated as live migration with numa without some functional tests, but at the same time, i guess if it turns out to have bugs then they just get fixed and we trust your integration testing downstream | |
| 15:12:56 | cfriesen | sean-k-mooney: will respin | |
| 15:13:12 | mriedem | in general i loathe landing anything related to pci/numa/sriov when we have no 3rd party integration testing for it | |
| 15:13:21 | cfriesen | we'll be doing integration testing for live migration too | |
| 15:13:35 | dansmith | mriedem: I too think it's fairly risky | |
| 15:13:41 | sean-k-mooney | cfriesen: ok it just need a bug added. | |
| 15:13:55 | sean-k-mooney | cfriesen: so it can be backported | |
| 15:14:02 | dansmith | I haven't really had time to go back through it from a holistic "how do these pieces fit together" sort of view | |
| 15:14:04 | artom | mriedem, I know :/ Which is why I want to replace the Intel CI with something that's actually maintained | |
| 15:14:17 | cfriesen | dansmith: mriedem: totally risky. but is it likely to break anything other than things that are already broken? | |
| 15:14:21 | dansmith | breaking live migration at all is a really bad thing to do | |
| 15:14:29 | dansmith | cfriesen: yes? | |
| 15:15:02 | artom | dansmith, basic live migration is tested in the gate though, right? | |
| 15:15:10 | mriedem | yes, | |
| 15:15:19 | dansmith | artom: during several partial upgrade situations? | |
| 15:15:31 | mriedem | so if worst case scenario you break live migration with numa instances, well, those were already broken anyway - and if you're using that config (once fixed) you can't even initiate it | |
| 15:15:34 | artom | No :( | |
| 15:15:40 | mriedem | dansmith: yes | |
| 15:15:44 | artom | Yes? | |
| 15:15:46 | mriedem | the live migration + grenade job | |
| 15:15:48 | dansmith | mriedem: we have one right? | |
| 15:15:50 | artom | \o/ | |
| 15:15:52 | mriedem | runs n-1 live migration back and forth | |
| 15:15:54 | mriedem | it's non-voting | |
| 15:16:13 | artom | We should *really* check its output on the NUMA LM patches then | |
| 15:16:15 | dansmith | mriedem: but not on systems that will report this stuff, even for non-numa flavors I think | |
| 15:16:24 | mriedem | https://review.openstack.org/#/c/637231/ | |
| 15:16:48 | mriedem | dansmith: correct we don't do any numa stuff in the gate | |
| 15:17:03 | sean-k-mooney | artom: i would start on the functional tests and we can add the whitebox tempest test after ff | |
| 15:17:04 | dansmith | it's not just numa-having instances, | |
| 15:17:33 | dansmith | it's that everyone's systems report numa stuff, and if any of this works on a numaless guest instance for CI but is weird on a real system... we won't have coverage of that | |
| 15:17:34 | mriedem | mrhillsman: would it be possible to use openlab for numa testing for nova? | |
| 15:17:37 | sean-k-mooney | artom: wether they are a blocker on not is seperate form if we should write them an i think we would all like to see functional tests for this eventually | |
| 15:18:41 | artom | sean-k-mooney, oh yeah, they'll be there | |
| 15:19:09 | dansmith | also, some amount of live migration of numa-having instances works today but just doesn't claim or do the right thing, right? | |
| 15:19:14 | artom | I'm wondering more "what gives others better confidence in the code to help it merge, automated integration tests that you have to run manually in your own env, or func tests?" | |
| 15:19:29 | dansmith | if we were to break that in a way that doesn't let people move their instances in an emergency, even if the numaness gets messed up, that's still a big problem I think | |
| 15:19:55 | mrhillsman | mriedem yes sir | |
| 15:20:28 | sean-k-mooney | dansmith: if the cpus are free on the destination then today everything works for pinned instance | |
| 15:20:37 | sean-k-mooney | similarly for hugepage instnace | |
| 15:20:39 | dansmith | right | |
| 15:21:04 | sean-k-mooney | the clamims get updated too i think | |
| 15:21:11 | dansmith | so if we break that in this because we didn't test it on real systems... that's not "the situation can't get worse" to me | |
| 15:21:25 | sean-k-mooney | they are either healed by the periodic task or done explicitly i cant remember | |
| 15:22:09 | mrhillsman | so long as we can provide what is needed. filling out https://github.com/theopenlab/openlab/issues/new?template=test-request.md with what is needed is required | |
| 15:22:47 | artom | sean-k-mooney, periodic task | |
| 15:24:26 | mriedem | mrhillsman: ah cool ok, i'll put this in our ptg agenda for discussion to see what minimum levels of testing we'd hope to get | |
| 15:24:27 | sean-k-mooney | mrhillsman: do numa testing we would need either nested virt on a host with at least 2 numa nodes or a dual numa beremetal host | |
| 15:24:32 | mriedem | mrhillsman: btw, who writes the tests? or is tempest used? | |
| 15:24:38 | cfriesen | sean-k-mooney: mriedem: hypervisor check bug is https://bugs.launchpad.net/nova/+bug/1818092 | |
| 15:24:39 | openstack | Launchpad bug 1818092 in OpenStack Compute (nova) "hypervisor check in _check_instance_has_no_numa() is broken" [Undecided,New] | |
| 15:25:31 | sean-k-mooney | cfriesen: cool can you add "Close-Bug: 1818092" to the commit in your patch | |
| 15:25:41 | cfriesen | yup, doing it now | |
| 15:25:46 | sean-k-mooney | awsome | |
| 15:25:52 | mrhillsman | mriedem you or some volunteers would | |
| 15:26:23 | sean-k-mooney | mriedem: we hope to move the redhat whitebox tempest plugin to openstack namespace | |
| 15:26:23 | artom | Unless it's like, in Perl or JS or something. | |
| 15:26:31 | mrhillsman | there is a lot of flexibility in how that works | |
| 15:27:36 | mrhillsman | i think adding to the ptg is good if it is not a priority right now | |
| 15:27:46 | mriedem | yup done L77 https://etherpad.openstack.org/p/nova-ptg-train | |
| 15:28:04 | mrhillsman | cool, i will be sure to be there | |
| 15:28:09 | mriedem | maybe chenrui's team can help a bit, i.e. Kevin_Zheng since he is familiar with nova | |
| 15:28:15 | mrhillsman | ++ | |
| 15:30:07 | openstackgerrit | Chris Friesen proposed openstack/nova master: fix up numa-topology live migration hypervisor check https://review.openstack.org/635350 | |
| 15:31:51 | cfriesen | sean-k-mooney: mriedem: there you go ^ | |
| 15:34:48 | mriedem | stephenfin: i saw your patch to deprecate config_drive_format for vfat | |
| 15:34:53 | mriedem | stephenfin: and it reminded me of this https://review.openstack.org/#/c/567860/ | |
| 15:35:12 | cfriesen | anyone know why requirements-check is failing for https://review.openstack.org/#/c/631363 ? It says "Package 'os-traits' is constrained to 0.4.0 which does not match the minimum version specifier 0.11.0 in requirements.txt" | |
| 15:35:17 | mriedem | stephenfin: i'm not 100% but i don't think we actually test live migration in the gate with a config drive on the instance | |
| 15:35:21 | bauzas | mriedem: yeah we agreed on following the deprecation path this morning | |
| 15:35:26 | mriedem | stephenfin: it would be good to have that wrinkle somewhere | |
| 15:35:33 | mriedem | bauzas: sure i understand, but that's not my point | |
| 15:35:36 | bauzas | ok | |
| 15:35:45 | mriedem | my point is we don't test live migration + config drive | |
| 15:35:47 | openstackgerrit | Merged openstack/nova master: conf: Call out where pci.alias should be set https://review.openstack.org/624445 | |
| 15:35:49 | mriedem | i don't think | |
| 15:36:19 | mriedem | so we might want to consider forcing config drive in our nova-live-migration job, | |
| 15:36:23 | mriedem | or the grenade live migration job | |
| 15:37:06 | mriedem | i think we also do everything with qcow2 image backend | |
| 15:37:11 | mriedem | and in https://review.openstack.org/#/c/567860/ i changed it to use raw | |
| 15:37:25 | mriedem | which is another wrinkle we don't really touch | |
| 15:39:23 | cfriesen | okay, why do we specify minimum package versions in both lower-constraints *and* requirements.txt? | |
| 15:40:10 | mriedem | tonyb: ^ | |
| 15:40:48 | jangutter | cfriesen: because the computer likes repetition because the computer likes repetition (paranoia reference) | |
| 15:42:11 | jangutter | cfriesen: I found this ( https://docs.openstack.org/project-team-guide/dependency-management.html ) a while ago and I'm not 100% sure. | |
| 15:42:13 | openstackgerrit | Chris Friesen proposed openstack/nova master: Add emulated TPM support to Nova https://review.openstack.org/631363 | |
| 15:42:14 | openstackgerrit | Chris Friesen proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files https://review.openstack.org/639934 | |
| 15:44:08 | sean-k-mooney | cfriesen: proably pinned in upperconstraitns | |