Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-21
17:37:01 tssurya sorry for the confusion
17:37:04 mdbooth mriedem: Incidentally lyarwood pointed out that we're not running volume tests in the live migration job by default, so on his suggestion I hacked a run based on another patch he pointed me at. Link is in a review comment.
17:38:44 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Update spec to reflect reality https://review.openstack.org/555000
17:39:28 stephenfin jaypipes, gibi: Could you stick this on your review queue. Just ensure the spec for a Queens feature matches the actual implementation https://review.openstack.org/555000
17:39:31 stephenfin *?
17:40:49 sean-k-mooney stephenfin: is that the pci numa policy stuff?
17:41:26 sean-k-mooney stephenfin: i taught we did not want to put the policy in the alias?
17:43:07 sean-k-mooney stephenfin: hum well its in the code here https://github.com/openstack/nova/blob/master/nova/pci/request.py#L90-L92 so i guess we did
17:43:15 cfriesen sean-k-mooney: ah, but we did...that way you can be flexible for one device and strict for another
17:43:28 openstackgerrit Dan Smith proposed openstack/nova stable/pike: Add --by-service to discover_hosts https://review.openstack.org/554603
17:45:01 sean-k-mooney cfriesen: yes i see that i guess it works but eventually i would hope we can get this kind of stuff out of configs and into openstack server create --device vendor-id=xyz ...
17:45:58 sean-k-mooney cfriesen: with cyborg becoming a thing that seams like the correct route long term
17:46:07 cfriesen sean-k-mooney: I'd be fine with that, but it might affect the instance packing (which dansmith has smacked me with before)
17:46:30 sean-k-mooney instance packing?
17:46:33 sean-k-mooney in placement
17:46:44 cfriesen defining flavors to pack instances neatly onto a compute node with minimal waste
17:47:15 dansmith cfriesen: I don't believe I smacked you, I just made a big farting noise and gave it a thumbs down
17:47:19 dansmith you know, to be respectfyul
17:47:21 sean-k-mooney cfriesen: ah ok. well currently passh through devices are not first class resouces. if they are it changes the usage model and billing model
17:49:02 sean-k-mooney but ya in any case having the numa policy in the alias kind of works but its not discoverable by the tenant so it wont work for everyone. espcilally the people that dont like setting things in the nova conf
17:50:42 openstackgerrit melanie witt proposed openstack/nova master: Add periodic task to clean expired console tokens https://review.openstack.org/325381
17:50:43 openstackgerrit melanie witt proposed openstack/nova master: Convert websocketproxy to use db for token validation https://review.openstack.org/333990
17:50:43 openstackgerrit melanie witt proposed openstack/nova master: Use ConsoleAuthToken object to generate authorizations https://review.openstack.org/325414
17:51:59 cfriesen sean-k-mooney: are you proposing that we add an HTTP API for PCI aliases?
17:53:16 sean-k-mooney cfriesen: well no i dont think we should keep adding features to the aliases but maybe in the flavour details show some of the alais fields
17:53:57 sean-k-mooney cfriesen: long term however i would much prefer this handeled with resouce requests and traits in the flavor then using aliases
17:55:02 sean-k-mooney cfriesen: the mater of how that resource request gets there(set by admin directly or via boot option e.g. --device) is a seperate disction
17:55:17 sean-k-mooney /disction/topic/
17:58:47 sean-k-mooney cfriesen: these https://github.com/openstack/nova/blob/master/nova/pci/request.py#L74-L89 can all be modeled as traits on the RP in placement and the numa_policy can be modeled as a flavor extra spec key that is consumed by the numa_topology/pci passhtrough filters.
18:04:19 openstackgerrit Mark Goddard proposed openstack/nova master: Request only instance_uuid in ironic node list https://review.openstack.org/539509
18:18:49 cfriesen once nova-compute spawns a privsep daemon, does it stay running after that?
18:21:13 sean-k-mooney cfriesen: yes and a seperate priv sep deamon is spawned for each context
18:22:23 sean-k-mooney cfriesen: so if you have two contexts with 2 different permission sets you will have 2 privsep deamons running for nova-compute
18:22:58 sean-k-mooney when nova compute exits the deamons also exits when the unix socket gets closed
18:26:15 melwitt dansmith: patch for removing the useless non-periodic task line has acks from efried_rollin and gibi https://review.openstack.org/#/c/554381
18:27:00 dansmith melwitt: ah yeah thanks for the reminder
18:27:08 dansmith melwitt: so confirmed what I was saying about that test yeah?
18:27:22 melwitt yes, I tried it out and indeed it was only covering that one line :
18:27:24 dansmith melwitt: and I also wonder if that was the only source of the overwriting-context problem initially
18:27:25 melwitt :\
18:27:29 dansmith ack
18:27:35 melwitt yeah, I think it was -_-
18:38:11 openstackgerrit Eric Berglund proposed openstack/nova master: PowerVM Driver: Snapshot https://review.openstack.org/543023
18:40:08 edmondsw https://review.openstack.org/#/c/547169/ should be a quick review if someone has a few minutes
18:40:44 edmondsw prereq to starting to flip some of those capabilities to True as we introduce functions in later commits
18:42:25 cfriesen sean-k-mooney: cool, thanks
18:46:08 mriedem dansmith: commented on mdbooth's change https://review.openstack.org/#/c/551302/
18:46:15 mriedem dansmith: overall it's ok, but i think we also have a problem in rollback
18:48:02 dansmith mriedem: cool thanks
18:53:56 mriedem i looked at his live migration test patch that enabled the volume-backed live migration tests,
18:54:04 mriedem the grenade live migration job passed, which uses mixed computes
18:54:16 mriedem and goes back and forth (rocky->queens->rocky)
18:54:20 mriedem and vice-versa
18:54:34 mriedem the other non-grenade live migration job failed, looks like all rpc messaging timeouts
18:57:04 mriedem huh, same thing in melwitt's patch to enable the volume-backed live migratoin tests http://logs.openstack.org/04/528104/6/check/legacy-tempest-dsvm-multinode-live-migration/ff90ecb/logs/subnode-2/screen-n-cpu.txt.gz?level=TRACE
18:57:25 mriedem i wonder if there is just something about doing a volume-backed live migration that takes that much longer such that pre_live_migration rpc call times out
18:58:19 melwitt yeah, I had been rechecking the patch periodically but it never passed reliably upon multiple rechecks
18:58:35 mriedem i'm seeing the same rpc timeouts in your patch and https://review.openstack.org/#/c/553377/
18:58:37 mriedem which depends on your patch
18:58:40 melwitt yeah
18:59:26 melwitt I just mean I had been checking on it to see if we were going to be able to re-enable those tests but it wasn't panning out. will try depending on your queens uca patch
18:59:29 mriedem you could try setting the rpc timeout to 120 in nova-cpu.conf in the job config and see if that makes a difference
18:59:44 mriedem i don't think libvirt is the problem here
18:59:51 melwitt okay, will do that instead
19:00:42 mriedem this is how you do something like that https://review.openstack.org/#/c/549789/9/playbooks/legacy/nova-cells-v1/run.yaml@38
19:00:55 mriedem except you'll use $NOVA_CPU_CONF
19:01:10 mriedem oh, but,
19:01:20 mriedem first you need to move the live migratoin job defs in tree
19:01:23 mriedem something i've been thinking about doing anyway
19:02:13 melwitt hm, okay. (on a call atm)
19:04:58 dansmith or I could finish my live heartbeating thing in oslo.messaging
19:09:52 mriedem or you could take this over https://review.openstack.org/#/c/452546/
19:10:00 mriedem like you PROMISED at the PTG
19:10:31 dansmith um, wut?
19:11:00 dansmith I thought you said you were going to do that?
19:11:20 mriedem oh on
19:11:23 mriedem *no
19:11:28 mriedem i said i had tried at one point and was shot down
19:11:35 mriedem and was welcome to others getting shot
19:12:15 mriedem btw i think the cellsv1 + neutron job is ready to go https://review.openstack.org/#/c/549789/
19:12:30 mriedem i can't drop the old cells v1 job from master until that flushes through
19:14:15 openstackgerrit Eric Berglund proposed openstack/nova master: PowerVM Driver: DiskAdapter parent class https://review.openstack.org/549053
19:14:34 openstackgerrit Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Localdisk https://review.openstack.org/549300
19:14:46 dansmith mriedem: that's a long ways from "promised" :)
19:15:03 mriedem i know, i was hoping the caps would convey the joke
19:15:38 mriedem like, "omfg there was pug shit EVERYWHERE!"
19:16:31 dansmith mriedem: so this patch's cells job came from the in-tree version https://review.openstack.org/#/c/549780/2 ?
19:16:58 dansmith ah, I guess there are two on there
19:18:21 dansmith mriedem: your job's cells-child log has a little extra red in it from the base job: http://logs.openstack.org/80/549780/2/check/nova-cells-v1/66d336e/logs/screen-n-cell-child.txt.gz?level=TRACE
19:18:31 dansmith although that looks vaguely familiar, so maybe not a problem
19:18:58 mriedem i've seen those before
19:19:03 mriedem race in the metadata updates
19:19:08 dansmith okay
19:19:23 mriedem so yeah what i did was move the existing job in-tree
19:19:28 mriedem and renamed it
19:19:34 mriedem b/c that's what we do when we move them in tree,
19:19:47 mriedem and then tweaked it to be the new thing with neutron
19:20:00 dansmith so after we merge that first one, we can remove the base job from infra, right?
19:20:06 mriedem in https://review.openstack.org/#/c/549789/ we are getting both jobs b/c at this point the legacy job is still in openstack-zuul-jobs
19:20:14 mriedem yeah i have all of thoes patches lined up

Earlier   Later