| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-14 | |||
| 11:36:06 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Change order of two classes https://review.opendev.org/689178 | |
| 11:36:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181 | |
| 11:36:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make '_wait_for_state_change' behave consistently https://review.opendev.org/689180 | |
| 11:36:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make 'ServerTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | |
| 11:42:57 | openstackgerrit | Stephen Finucane proposed openstack/nova master: DNM: Try to figure out why rbd import fails https://review.opendev.org/694260 | |
| 12:22:25 | openstackgerrit | Stephen Finucane proposed openstack/nova master: DNM: python3 experiments https://review.opendev.org/694281 | |
| 13:26:05 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Boot from volume instance rescue https://review.opendev.org/694063 | |
| 13:26:05 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Virtual instance rescue with stable disk devices https://review.opendev.org/693849 | |
| 14:19:27 | efried | o/ nova | |
| 14:19:39 | efried | Is anyone around for the meeting? | |
| 14:20:00 | efried | First meeting since PTG+USADST, so an hour earlier for many of us. | |
| 14:23:48 | gibi_off | efried: I have to skip it today | |
| 14:24:16 | efried | Okay. You and bauzas and stephenfin were going to be the stars of the show, so... | |
| 14:24:35 | efried | and next week's (at 2100 utc) is going to be inconvenient for all of you. | |
| 14:24:41 | bauzas | sorry, on 1:1 meeting now | |
| 14:26:16 | gibi_off | efried: yeah 2100 is bad, but today (and yesterday) I have to be present on an internal conference demoing the bandwidth stuff repeatedly | |
| 14:26:37 | efried | that sounds like more fun than a nova meeting anyway. | |
| 14:26:59 | gibi_off | efried: and as this is done in an open demo area I don't know when somebody pops up at our booth | |
| 14:27:37 | gibi_off | I can imagine there is a way to define fun that applies to me now :) | |
| 14:27:38 | efried | Heh, hang an "out to lunch" placard :P | |
| 14:27:46 | gibi_off | :) | |
| 14:34:39 | stephenfin | efried: I got confused with the timezones. Drat /o\ | |
| 14:34:46 | stephenfin | DST-- | |
| 14:43:45 | mriedem | huh, i forgot to change my calendar as well :) | |
| 14:48:52 | mriedem | efried: was there anything you wanted to go over? my thing in open discussion is in the ML and i'm just going to do a quick PoC I think | |
| 14:49:02 | mriedem | there were a few stable branch releases | |
| 14:49:08 | mriedem | and elod was added to the nova-stable-maint team | |
| 14:52:22 | efried | mriedem: The other open (which luyao hadn't put on the agenda as of meeting start time) was whether the LM bits of vPMEM need a BP/spec or not. What are your thoughts? | |
| 14:55:20 | mriedem | my gut reaction is vPMEM hasn't even baked to the point of thinking about supporting live migration | |
| 14:55:40 | mriedem | i.e. i don't think we even support live migration with vgpus do we? | |
| 14:55:48 | mriedem | and people might actually be using that by now | |
| 14:56:12 | mriedem | vpmem and live migration is likely complicated that a short spec probably doesn't hurt | |
| 14:56:20 | mriedem | *complicated enough | |
| 14:56:45 | mriedem | stephenfin: i commented on https://review.opendev.org/#/c/692374/ | |
| 14:57:02 | mriedem | tl;dr i don't think USE_PYTHON3=True is making it to the devstack-plugin-ceph run when the post-test script runs it | |
| 14:57:36 | mriedem | stephenfin: also, in trying to convert the nova-live-migration job to zuulv3 here https://review.opendev.org/#/c/693364/ oi | |
| 14:57:53 | mriedem | i'm coming to the conclusion that it would be a ton simpler to just split the nova-live-migration job in two, | |
| 14:58:06 | mriedem | one for block migration and local disk and one with ceph + shared disk | |
| 14:58:14 | efried | mriedem: Pretty sure the impl is already done, it just didn't make Train. Anyway, I agree it could do with a short spec, as I think it needs new object fields, so we should at least be able to bikeshed the names of those. <== luyao | |
| 14:58:35 | mriedem | efried: is the vpmem 3rd party ci stable? | |
| 14:58:46 | mriedem | and if it's stable, can they run it multinode to test live migration? | |
| 14:58:50 | efried | dunno. alex_xu luyao ^ ? | |
| 14:58:51 | mriedem | at least periodically? | |
| 14:59:18 | efried | I thought the CI was stable at the end of train, for single node. Don't know the status of LM CI. | |
| 14:59:38 | mriedem | i just don't have a ton of energy to invest in building feature functionality on top of a thing that i don't know if anyone is using it | |
| 14:59:50 | mriedem | like SEV in that way... | |
| 15:00:21 | mriedem | having said all that, i think dansmith would be totally on board with building in live migration support for vpmems, that's right up his alley | |
| 15:01:01 | efried | I think of it less as "tack a feature onto this thing" and more as "piece of the puzzle that didn't quite get finished with the rest". | |
| 15:02:16 | efried | oh, maybe I'm wrong. I see the patch wasn't proposed until Oct. | |
| 15:02:32 | mriedem | so looking at a recent pmem ci run http://52.27.155.124/79/694179/2/check/pmem-tempest-plugin-filtered/1123b99/job-output.txt | |
| 15:02:36 | mriedem | it runs 3 tests in less than a minute | |
| 15:02:43 | mriedem | the entire job takes about 30 minutes | |
| 15:03:05 | mriedem | surely there is more testing that could be done with operations on a server with pmems, though i'm not sure what test_server_basic_ops does | |
| 15:03:24 | mriedem | e.g. if resize works, cold migration should work, so maybe first steps are making that ci job multinode and testing cold migrate | |
| 15:04:05 | sean-k-mooney | looking at http://ciwatch.mmedvede.net/project?project=nova it seams somewhat statble although it is missing some patches | |
| 15:04:41 | mriedem | ok here is the basic ops test https://github.com/LuyaoZhong/vpmem-tempest-plugin/blob/master/vpmem_tempest_plugin/tests/scenario/test_server_basic_ops.py#L127 | |
| 15:04:56 | mriedem | create a server, ssh into it, delete the server... | |
| 15:05:03 | stephenfin | mriedem: https://review.opendev.org/#/c/694281/1 | |
| 15:05:03 | mriedem | so reboot + pmem isn't being tested | |
| 15:05:06 | mriedem | shelve/unshelve | |
| 15:05:16 | stephenfin | I think that fixes the Python 3 issues, right? | |
| 15:05:23 | stephenfin | Or have I missed something obvious? | |
| 15:06:16 | sean-k-mooney | mriedem: i suspect pause/unpause and suspend/resume are also missing | |
| 15:06:37 | stephenfin | obviously converting things to zuul v3 native would be better though, of course, but maybe I should squash that back so we can continue working on the Python 3 only'ification in parallel | |
| 15:07:17 | mriedem | stephenfin: looks like it does yeah, py3 is being used here https://zuul.opendev.org/t/openstack/build/2850353a84da4ec08b4f7cda7a08a781/log/logs/subnode-2/screen-n-cpu.txt.gz#3 | |
| 15:07:20 | sean-k-mooney | mriedem: they have a resize test https://github.com/LuyaoZhong/vpmem-tempest-plugin/blob/master/vpmem_tempest_plugin/tests/scenario/test_server_pmem_ops.py#L105 | |
| 15:07:27 | mriedem | sean-k-mooney: that would be resize to the same host though | |
| 15:07:32 | stephenfin | \o/ | |
| 15:07:38 | sean-k-mooney | yes | |
| 15:07:54 | mriedem | stephenfin: so you want to squash that into https://review.opendev.org/#/c/692374/ ? | |
| 15:08:05 | stephenfin | yeah | |
| 15:08:08 | mriedem | go for it | |
| 15:08:08 | sean-k-mooney | but if resize on same host work then they should be able to resize to a different host no? if it was run multinode | |
| 15:08:13 | stephenfin | though maybe I should fix the devstack plugin first? | |
| 15:08:28 | mriedem | stephenfin: mostly the cinder core team | |
| 15:08:41 | mriedem | https://review.opendev.org/#/admin/groups/1196,members | |
| 15:08:45 | stephenfin | okay, they'll want this too so | |
| 15:08:52 | stephenfin | I'll fix that and add depends-on | |
| 15:09:09 | luyao | mriedem, efried : VPMEM CI is stable, but we haven't LM test yet. | |
| 15:09:30 | mriedem | the zuulv3 conversion of nova-live-migration gets super wonky with that post-test-script because the script assumes devstack-gate and still requires that for the nova-multinode-grenade job | |
| 15:09:57 | mriedem | so that's kind of why i'm tempted to convert nova-live-migration to zuulv3 and split it into two jobs, one with -ceph | |
| 15:10:04 | efried | luyao: I agree with mriedem that it would be good to expand the CI with more scenarios testing the existing code before adding LM support, see scrollback. | |
| 15:10:23 | mriedem | luyao: could you start by making the pmem ci job multinode and adding a test for cold migrate? | |
| 15:10:43 | mriedem | and shelve/unshelve | |
| 15:10:48 | stephenfin | Yeah, I spent the bones of a day pre-PTO trying to convert it and got nowhere so I'm glad you're working on it 😅 | |
| 15:10:57 | stephenfin | Splitting it up makes sense to me | |
| 15:11:08 | mriedem | it was pretty easy until i got to the script | |
| 15:11:10 | mriedem | which is a nightmare | |
| 15:12:32 | luyao | mriedem, efried : OK, I will start preparing more CI tests | |
| 15:13:42 | efried | luyao: So I still think because of the RPC (and therefore upgrade) impacts, we should definitely have a short spec for LM. In that spec, we should state that moving forward with LM will be contingent on first demonstrating CI coverage for $stuff above. | |
| 15:14:32 | luyao | efried: get it, thanks | |
| 15:15:23 | huaqiang | sean-k-mooney: about the per-vm-pci-NUMA-policy spec, I want to confirm that it defines a policy applied to all PCI devices for whole VM, right? | |
| 15:15:26 | gibi_off | eandersson: hi! have you opened a bug for the false error log in case of https://github.com/openstack/nova/commit/a5269012a3b442a9e4055a7d523faff45f105f2b#diff-77f9348ab09642ba46409b6828af4af0R1327 being hit on an empty compute? | |
| 15:15:34 | stephenfin | Dumb question, but does Ubuntu allow you to install e.g. 'python-foo' and 'python3-foo' side-by-side without them stomping on each other? | |
| 15:15:49 | efried | stephenfin: shore | |
| 15:15:56 | stephenfin | kewl | |
| 15:16:02 | sean-k-mooney | huaqiang: yes | |
| 15:16:16 | sean-k-mooney | huaqiang: and it takes precidnce over the alias in the config | |
| 15:16:48 | sean-k-mooney | so it applies to both neutron sriov ports and to flavor based pci passthough | |
| 15:17:10 | huaqiang | sean-k-mooney: then it is impossible to set three and more than three NUMA policies to PCI devices in a VM | |