Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-14
15:01:01 efried I think of it less as "tack a feature onto this thing" and more as "piece of the puzzle that didn't quite get finished with the rest".
15:02:16 efried oh, maybe I'm wrong. I see the patch wasn't proposed until Oct.
15:02:32 mriedem so looking at a recent pmem ci run http://52.27.155.124/79/694179/2/check/pmem-tempest-plugin-filtered/1123b99/job-output.txt
15:02:36 mriedem it runs 3 tests in less than a minute
15:02:43 mriedem the entire job takes about 30 minutes
15:03:05 mriedem surely there is more testing that could be done with operations on a server with pmems, though i'm not sure what test_server_basic_ops does
15:03:24 mriedem e.g. if resize works, cold migration should work, so maybe first steps are making that ci job multinode and testing cold migrate
15:04:05 sean-k-mooney looking at http://ciwatch.mmedvede.net/project?project=nova it seams somewhat statble although it is missing some patches
15:04:41 mriedem ok here is the basic ops test https://github.com/LuyaoZhong/vpmem-tempest-plugin/blob/master/vpmem_tempest_plugin/tests/scenario/test_server_basic_ops.py#L127
15:04:56 mriedem create a server, ssh into it, delete the server...
15:05:03 stephenfin mriedem: https://review.opendev.org/#/c/694281/1
15:05:03 mriedem so reboot + pmem isn't being tested
15:05:06 mriedem shelve/unshelve
15:05:16 stephenfin I think that fixes the Python 3 issues, right?
15:05:23 stephenfin Or have I missed something obvious?
15:06:16 sean-k-mooney mriedem: i suspect pause/unpause and suspend/resume are also missing
15:06:37 stephenfin obviously converting things to zuul v3 native would be better though, of course, but maybe I should squash that back so we can continue working on the Python 3 only'ification in parallel
15:07:17 mriedem stephenfin: looks like it does yeah, py3 is being used here https://zuul.opendev.org/t/openstack/build/2850353a84da4ec08b4f7cda7a08a781/log/logs/subnode-2/screen-n-cpu.txt.gz#3
15:07:20 sean-k-mooney mriedem: they have a resize test https://github.com/LuyaoZhong/vpmem-tempest-plugin/blob/master/vpmem_tempest_plugin/tests/scenario/test_server_pmem_ops.py#L105
15:07:27 mriedem sean-k-mooney: that would be resize to the same host though
15:07:32 stephenfin \o/
15:07:38 sean-k-mooney yes
15:07:54 mriedem stephenfin: so you want to squash that into https://review.opendev.org/#/c/692374/ ?
15:08:05 stephenfin yeah
15:08:08 mriedem go for it
15:08:08 sean-k-mooney but if resize on same host work then they should be able to resize to a different host no? if it was run multinode
15:08:13 stephenfin though maybe I should fix the devstack plugin first?
15:08:28 mriedem stephenfin: mostly the cinder core team
15:08:41 mriedem https://review.opendev.org/#/admin/groups/1196,members
15:08:45 stephenfin okay, they'll want this too so
15:08:52 stephenfin I'll fix that and add depends-on
15:09:09 luyao mriedem, efried : VPMEM CI is stable, but we haven't LM test yet.
15:09:30 mriedem the zuulv3 conversion of nova-live-migration gets super wonky with that post-test-script because the script assumes devstack-gate and still requires that for the nova-multinode-grenade job
15:09:57 mriedem so that's kind of why i'm tempted to convert nova-live-migration to zuulv3 and split it into two jobs, one with -ceph
15:10:04 efried luyao: I agree with mriedem that it would be good to expand the CI with more scenarios testing the existing code before adding LM support, see scrollback.
15:10:23 mriedem luyao: could you start by making the pmem ci job multinode and adding a test for cold migrate?
15:10:43 mriedem and shelve/unshelve
15:10:48 stephenfin Yeah, I spent the bones of a day pre-PTO trying to convert it and got nowhere so I'm glad you're working on it 😅
15:10:57 stephenfin Splitting it up makes sense to me
15:11:08 mriedem it was pretty easy until i got to the script
15:11:10 mriedem which is a nightmare
15:12:32 luyao mriedem, efried : OK, I will start preparing more CI tests
15:13:42 efried luyao: So I still think because of the RPC (and therefore upgrade) impacts, we should definitely have a short spec for LM. In that spec, we should state that moving forward with LM will be contingent on first demonstrating CI coverage for $stuff above.
15:14:32 luyao efried: get it, thanks
15:15:23 huaqiang sean-k-mooney: about the per-vm-pci-NUMA-policy spec, I want to confirm that it defines a policy applied to all PCI devices for whole VM, right?
15:15:26 gibi_off eandersson: hi! have you opened a bug for the false error log in case of https://github.com/openstack/nova/commit/a5269012a3b442a9e4055a7d523faff45f105f2b#diff-77f9348ab09642ba46409b6828af4af0R1327 being hit on an empty compute?
15:15:34 stephenfin Dumb question, but does Ubuntu allow you to install e.g. 'python-foo' and 'python3-foo' side-by-side without them stomping on each other?
15:15:49 efried stephenfin: shore
15:15:56 stephenfin kewl
15:16:02 sean-k-mooney huaqiang: yes
15:16:16 sean-k-mooney huaqiang: and it takes precidnce over the alias in the config
15:16:48 sean-k-mooney so it applies to both neutron sriov ports and to flavor based pci passthough
15:17:10 huaqiang sean-k-mooney: then it is impossible to set three and more than three NUMA policies to PCI devices in a VM
15:17:57 huaqiang and we have three PCI numa policies in total
15:17:59 sean-k-mooney if you mean you cant have different policies for different devices correct
15:18:35 huaqiang yes, that is I want to confirm
15:18:40 sean-k-mooney that is why its titled vm-scoped-sriov-numa-affinity
15:19:17 sean-k-mooney if we want to do interface scoped sriov polices we still can as a seperate qos policy on the neuton port
15:19:39 sean-k-mooney i would like to enable that too at some point but after this initall effort is done once we see if its required
15:20:23 luyao efried, mriedem: I have another question for "track error mirations and orphans in resource tracker", I think it is a bug, which have some effects on vpmem feature. Do I need put all vpmem live migration stuff based on the bug fixed. I'm not sure how to address this bug at the moment. Could you help me figure it out, I sent an emaila few days ago.
15:22:44 sean-k-mooney luyao: i think those are two different topics
15:22:53 luyao I put my solution on this etherpad. I hope it is clear for you. We can discuss on the etherpad. https://etherpad.openstack.org/p/track-err-migr-and-orphans-in-RT
15:23:01 efried luyao: I'll try to look, but it sounds daunting. If you think there's a bug (vpmem notwithstanding) we would welcome a patch to fix it.
15:24:32 sean-k-mooney efried: part of it is a design choice. which is been adressed in a bugfix already
15:25:09 huaqiang sean-k-mooney: Got. thanks
15:25:16 sean-k-mooney specificly there is a patch to reap orpahanded vms that have a nova metada in there domain xml but were deleted in the db
15:25:22 luyao efried, sean-k-mooney : so I can do the vpmem live migration and bug fix parallelly,right?
15:25:33 sean-k-mooney as an extention to the existing periodic task
15:25:42 efried mriedem: https://review.opendev.org/#/c/694248/ FYI (wait to recheck patches failing requirements check until ^ merges)
15:25:52 efried (per smcginnis)
15:26:14 sean-k-mooney luyao: provided you dont need to modify the same code sure even then you can factor out the common code into a shared patch
15:26:33 sean-k-mooney luyao: from a review persepective it will be simpler if they are seperate
15:27:23 openstackgerrit Matt Riedemann proposed openstack/nova master: docs: update SUSPENDED server status wrt supported drivers https://review.opendev.org/694329
15:27:41 mriedem efried: i'm not sure what you're showing me
15:28:03 mriedem is that related to why my sqla-migrate change failed?
15:28:07 efried mriedem: the reqs job is broken until that merges. We've got a couple patches in the pipe that keep failing on reqs. yes.
15:28:25 mriedem ack, thanks
15:30:46 luyao sean-k-mooney: OK, get it, they should be separate if there is no surprise. :)
15:44:20 mriedem sean-k-mooney: did you ever write a functional test for trying to delete a compute service while a server is in VERIFY_RESIZE status for the host? i vaguely recall that coming up when i wrote test_evacuate_then_delete_compute_service
15:45:30 openstackgerrit Stephen Finucane proposed openstack/nova master: Convert legacy nova-live-migration and nova-multinode-grenade to py3 https://review.opendev.org/692374
15:45:48 sean-k-mooney am no i rememebr that but i think i ended up getting pulled into something else
15:46:41 sean-k-mooney i think we were conserned the resize confim would fail if the source was deleted right
15:47:39 sean-k-mooney because we would not be able to do the source host clean up
15:47:50 mriedem likely yes
15:47:57 mriedem or same if the dest is deleted and you try to revert
15:48:08 sean-k-mooney ya
15:48:19 mriedem ok i'll just write those tests today
15:49:31 sean-k-mooney ill freely admint while is am begining to understand the functional test better of late i still am less comfortable writing them then i woudl like to be
15:49:52 mriedem copy/paste/modify
15:50:25 mriedem that + stackforge = highly paid professional software engineer
15:50:29 sean-k-mooney ya so that i can do but im trying to understand some of the fixture and how they related a llitle more os if i need to write one i can
15:50:50 mriedem well, the best way to learn is to get started
15:51:18 sean-k-mooney hehe which is a good segway for me to go back to writign the ones you asked for yesterday
15:52:44 sean-k-mooney when i do have them working they are a nice way to test the code. bauzas has been suggestign we try to do more test driven developemnt usign functional tests
15:53:54 sean-k-mooney i do tend to rely on deploying things on really hardware or nested vert more then i proably should in my intally dev sometimes
15:56:03 sean-k-mooney bauzas: might also be motivated by a lack of hardware to test his things
15:59:17 openstackgerrit Lee Yarwood proposed openstack/nova-specs master: Boot from volume instance rescue https://review.opendev.org/694063
16:00:27 openstackgerrit Stephen Finucane proposed openstack/nova master: Stop testing Python 2 https://review.opendev.org/687954
16:06:37 openstackgerrit Stephen Finucane proposed openstack/nova master: Convert legacy nova-live-migration and nova-multinode-grenade to py3 https://review.opendev.org/692374
16:06:38 openstackgerrit Stephen Finucane proposed openstack/nova master: Stop testing Python 2 https://review.opendev.org/687954

Earlier   Later