Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-07
16:56:52 efried is that what does it?
16:57:08 efried I'll try that
16:57:26 mriedem efried: dansmith: so PS9 might work, but i've seen reno complain about links in weird ways between sections, so we'll see if the link defined in the first section is honored in the second
16:57:45 dansmith mriedem: efried said it was only needed once
16:57:58 mriedem i guess we'll know in a couple of hours :)
16:58:04 efried should be the case, but it depends how reno does its build.
16:58:07 stephenfin Sphinx builds doctrees and reuses those when possible. Unfortunately it occasionally breaks if you e.g. remove a document that previously existed (like by changing to an older branch)
16:58:13 stephenfin probably less on an issue for renos
17:01:19 efried stephenfin: In the specs repo I wrote a tool that would only build one directory at a time. It was a fiddly thing to figure out exactly how to word the sphinx-build command. In this case... if I could restrict it to building just $release.rst it would probably be equivalent.
17:09:42 efried dansmith, mriedem: local build is green, the link works in both spots as expected. Fast approving.
17:10:11 efried ...mriedem do you agree with the new wording? Or at least it's not worse?
17:12:06 efried dansmith: we shouldn't have to backport the fixture poison beyond Train, right? because we wouldn't be backporting any future fix that hasn't gone through that fixture in Train+
17:12:25 dansmith efried: we wouldn't backport that anyway
17:14:01 openstackgerrit Dan Smith proposed openstack/nova stable/train: Fix instance.hidden migration and querying https://review.opendev.org/706582
17:14:15 dansmith backport ^
17:15:50 efried +1
17:17:40 efried stephenfin: fwiw commenting out the 'rm' doesn't seem to be speeding anything up. It's still scanning the world, which seems to be the part that takes forever.
17:17:52 stephenfin darn
17:17:59 stephenfin I was thinking all that would be cached
17:18:24 efried stephenfin: considering a target that patches index.rst and (re)moves $not-this-release.rst before building.
17:18:30 gibi efried, dansmith: do we have re-schedule handling implemented in the cyborg series?
17:18:42 efried ...and then of course it would have to undo that after
17:18:58 dansmith gibi: unsure, I'm kinda depending on sean-k-mooney to poke at some of those things
17:19:10 efried gibi: How is that handled for bw? I would have thought it would be the same way.
17:19:33 stephenfin efried: Have we closed any more branches yet? https://github.com/openstack/nova/commit/857b5003ccc0b37f4642ed77d9f0d08f9ee28dfb
17:19:33 dansmith yeah, I think it should be the same for ports even, but actually showing that it works is different of course
17:19:55 stephenfin I guess with EM we no longer do that
17:19:58 efried stephenfin: yeah, was thinking about that
17:20:13 efried do we truly never EOL anything at this point?
17:20:21 gibi the arqs needs to be unbound from the failed dest host and bound to the alternate host. the placement handling shoudl work out of the box, but we at least needs some test coverage to prove
17:21:28 stephenfin Maybe we could just live with the lack of updated release notes for those branches?
17:21:38 dansmith gibi: yup
17:22:24 sean-k-mooney dansmith: how would you like me to test reschdule? i can set up a multi node deployment i guess and maybe kill libvirt?
17:22:31 efried gibi: agree. At a glance, it's weird that delete_arqs_for_instance is first introduced here https://review.opendev.org/#/c/673735/
17:24:06 dansmith sean-k-mooney: I dunno.. if that'd work then maybe that's easy, or just throw a raise into the code on one node?
17:24:43 gibi sean-k-mooney: I think if you kill libvirt then the compute service will go down and the scheduler will not select it as a target
17:24:52 efried stephenfin: I removed ocata-train rsts and from index, it's still taking bloody forever. It's the scanning that's slow.
17:25:28 stephenfin efried: dhellmann told me there was a way to configure earliest-version to previous each one going back forever
17:25:29 sean-k-mooney if i do a multinode deployment then maybe i could kill ovs or the cyborg agent instead
17:25:33 dansmith gibi: well, if you disable the compute filter you can make that work still I think
17:25:35 gibi Sundar, dansmith, efried: anyhow I left a -1 to get some answer / re-schedul test coverage from Sundar https://review.opendev.org/#/c/631244/61/nova/tests/functional/test_servers.py@7625
17:25:37 sean-k-mooney so that the vif plugin would fail
17:25:39 efried stephenfin: I imagine it'd be a matter of fixing the reno.sphinxext code itself
17:25:52 dansmith gibi: ack
17:25:54 gibi dansmith: true, that can be done
17:26:43 gibi another question, does the series gracefully handle that move operations are not supported yet with accelerators?
17:26:55 stephenfin efried: I think earliest-version would do the trick. Let me try with that
17:29:09 dansmith gibi: probably not, but I'm trying to think of other examples of things we know don't work where we handle that gracefully
17:29:32 gibi dansmith: qos ports was implemented a check at the API level
17:29:34 dansmith gibi: there are some cases where we don't really know until the virt driver(s) get involved, which is *kinda* this case
17:30:07 gibi dansmith: here we know that nova does not support migrating an instance with arq yet, so we should reject that
17:30:11 dansmith gibi: okay but that's kindof a high-level thing because the coordination needs to be done above the compute (i.e. with neutron) anyway right?
17:30:29 dansmith gibi: you mean libvirt I assume
17:31:01 stephenfin efried: nope, still a couple of minutes and fans running at full blast :)
17:31:34 dansmith well, hmm, I was going to say the nova bits would still try, but maybe not because the conductor kicks the bind.. /me looks
17:31:40 gibi I think not just libvirt but also nova needs to grow support for migration with arqs, like re-querying the resource request of the arq at the start of the migration to include them into the scheduling request
17:32:52 dansmith gibi: yeah I guess you're right since we don't persist those in the reqspec
17:33:32 gibi OK, left this as well as a comment in https://review.opendev.org/#/c/631244/61
17:35:06 gibi I have to leave soon I will continue reading the series on Monday
17:35:34 dansmith I guess we probably have cold migration handling for pci devices specifically,
17:35:57 dansmith but I was kinda thinking this would fail for live migration like I think it does for pci, which is.. late in the virt driver IIRC
17:37:37 gibi my point is that we know we need to write some nova code to support these ops. So while we dont have that code we can reject such ops from the API.
17:38:21 dansmith gibi: yep, agree, I'm just talking out loud
17:38:30 dansmith talking out loud? thinking out loud :)
17:39:18 dansmith I don't want to put api-level barriers to something that is just a virt limitation but you're right, none of the paths where we do the arq stuff gets tickled in the, for example, resize paths
17:40:11 gibi yeah, I thin resize, migrate, live migrate, evacuate, unshelve (after offload) needs some code to recreate the proper resource request for the scheduling
17:40:41 dansmith yup
17:42:09 dansmith gibi: I was telling efried the other day that I had been so tunnel-vision on the bones of this that I wanted him to run through it again to break that up for me
17:42:18 dansmith for exactly this reason, so thanks for being that force :)
17:44:39 gibi I saw that two of you already handling this seris so I decided to focus energies elsewhere. but agree that fresh eyes helps to se things differently
17:48:16 dansmith yup
18:10:41 spatel sean-k-mooney: I am back now, sorry was in back to back meeting
18:38:05 sean-k-mooney dansmith: we have a check that blocks live migration if there are pci_request spec object that are not related to neutorn sriov ports
18:38:18 sean-k-mooney dansmith: but that wont block cyborg devices
18:38:25 dansmith sean-k-mooney: okay so something similar for cyborg I guess
18:38:31 sean-k-mooney since we also dont have pci_request spec objects
18:39:04 sean-k-mooney well for now we could just check if the flavor has accl:device-profile or whatever teh extraspec is
18:39:18 dansmith right
18:40:22 sean-k-mooney im not sure we will be able to do that check in the api because we would need to check the embeded flaovr form the cell db. actully no we can. we can grab it from the request_spec in the api db
18:40:42 sean-k-mooney so ya we can reject the live migration in the api layer until we support that
18:41:07 dansmith not sure what the problem is.. the api can look at the instance's embedded flavor
18:41:30 sean-k-mooney yep it can so no problem
18:41:45 sean-k-mooney i was thinking we might need to do a down call to the cell db to get it but we dont
18:41:49 sean-k-mooney so all good
18:42:06 dansmith we do to get the actual embedded flavor, but that's fine of copurse
18:42:21 dansmith anything that does anything on an instance gets the instance record from the cell in the api
18:42:28 sean-k-mooney well the embeded flavor is aslo stored in the api db in the request spec
18:42:53 dansmith it's stored as it was at the time of creation, but not necessarily the same as what the instance has now
18:43:04 dansmith and if we've done a data migration or something they could have diverged
18:43:11 sean-k-mooney oh that wont be update after reisze?
18:43:18 sean-k-mooney ah ok
18:43:25 dansmith the instance's actual flavor is what we should use, and it's no more expensive to get
18:43:44 sean-k-mooney sure makes sense
18:45:16 mriedem the requestspec.flavor is updated as part of a resize, but .... there be bugs
18:50:04 spatel sean-k-mooney: here is the virsh capability - http://paste.openstack.org/show/789301/
18:51:00 spatel here is the vCPU pinning map - http://paste.openstack.org/show/789302/
18:58:23 sean-k-mooney ok so the first 15 cpus are pinned to host numa node 0
18:58:33 spatel Yes
18:58:59 sean-k-mooney and the second 15 are all pinned to host numa node 1
18:59:02 sean-k-mooney so this is correct

Earlier   Later