Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-02-12
15:45:00 kashyap mdbooth: If you get some time, I'd really appreciate some libvirt driver and sensible eyes in general on this: https://review.openstack.org/#/c/632507/
15:45:18 kashyap mdbooth: I know it not fully complete, but the serial / char dev code is tripping me up a bit.
15:46:01 kashyap (Funnily enough, for me, the bug that is referenced in the code that's being deleted ... was a bug I filed 5 years ago)
15:46:11 sean-k-mooney in the sriov case if the RP name was the PF adress neutron and nova would agree on it at least
15:46:59 sean-k-mooney but really that is just "bit stealing" rather and a convention rather then a dedicated mechanium
15:48:15 jaypipes sean-k-mooney: that's basically what gibi's doing in https://review.openstack.org/#/c/623543/23/nova/compute/manager.py@2138
15:48:47 efried sean-k-mooney: We already do that in the vgpu code too. It doesn't help, because the allocation doesn't include the rp name.
15:48:53 sean-k-mooney i mean it will work as long as the service that need to share an RP agree on on how to chose the name
15:49:01 jaypipes the issue is that I see that as a fragile solution that depends on the name structure being codified and not changing ever. as soon as the device naming convention changes, everything would blow up.
15:49:24 cdent ++
15:49:25 sean-k-mooney efried: well you could lookup the rp by its uuid if you need the info form the allocation
15:49:28 efried shrug, be like saying if the mapping algorithm changes everything blows up
15:49:37 efried sean-k-mooney: This is where we started.
15:49:41 efried sean-k-mooney: Where do you look up the rp?
15:49:52 sean-k-mooney in placement
15:49:55 efried in placement? no, because we've said we don't want virt talking directly to placement
15:50:06 efried in the provider tree? Sure, but you have to get the provider tree somehow
15:50:14 jaypipes providers should always be *externally referenced* (i.e. any time two services need to exchange info about a provider) using a UUID. internally, within a service, the service should be responsible for caching an internal name -> UUID map.
15:50:19 efried in a mapping? Sure, but the mapping needs to be saved by the virt driver
15:50:27 sean-k-mooney if the virt dirver create the RP with a specific name in know it could take the uuid for the allcoation and look it up and get the name back
15:50:49 efried by talking to placement? No, see above. By looking in a mapping? Sure, see above.
15:51:04 sean-k-mooney efried: it only need to be saved for efficency.
15:52:19 efried sean-k-mooney: The proposed solution is for update_provider_tree to save a mapping of RP UUID to <something> and then for spawn to retrieve that information. The easiest thing would be for update_provider_tree simply to save the provider_tree as an instance var on the virt driver. Then spawn can just ask that guy for the provider with a given uuid and get the name. Since the name is already generated by convention, it can use it
15:53:04 sean-k-mooney efried: sure that makes sense
15:53:11 efried if we really don't like using the naming convention, upt could save some other arbitrarily-formatted dict as an instance var, to the same end.
15:53:52 sean-k-mooney but making that internal mapping to the uuid stable it the respociblity of the service consuming placment in that case not placement
15:54:16 efried yes, update_provider_tree is that thing in either case.
15:54:49 efried even if spawn got to request the provider record from placement and use the naming convention, it was still upt's responsibility to name the provider appropriately.
15:55:06 sean-k-mooney ya
15:55:17 efried but I prefer the idea of upt caching^wahem, saving something as an instance var.
15:55:53 sean-k-mooney isnt this partly solved by https://review.openstack.org/#/c/612497/ also
15:56:03 sean-k-mooney the provider ymal file
15:56:19 efried sean-k-mooney: It could be, but I kind of hate the idea of making it the operator's responsibility
15:56:53 efried as long as there's reasonable default behavior if that file is missing (or composed poorly) it would be okay.
15:56:54 sean-k-mooney well there is no reason the virtdriver could not generate it on first boot if its not provided
15:57:03 sean-k-mooney and then operator could add to it
15:57:17 efried also ew.
15:57:28 sean-k-mooney that said that might mess with som config managment stuff
15:57:30 efried Now what if I hot-(un)plug devices or otherwise change resources.
15:57:33 efried yeah
15:57:56 efried now I'm forced to hand-edit the file after that first time. Or remember to blow it away so it can be regenerated, but then I lose my customizations.
15:58:03 sean-k-mooney you first update teh yaml file then send a sig hup to the compute agent then you remove the hardware
15:58:35 efried as opposed to just... removing the hardware and everything works.
15:59:01 sean-k-mooney that is not really a thing today
15:59:14 efried neither is generic numa affinity
15:59:18 sean-k-mooney i mean i "might" work in some cases
15:59:45 efried Seems like all of this is stuff we're striving for.
16:00:16 efried The whole reason update_provider_tree happens on a loop is so that it can, in the future, automatically adjust the provider tree when stuff like this happens.
16:00:27 sean-k-mooney i think https://etherpad.openstack.org/p/nova-ptg-train everyting from lin 7-30 is very related
16:01:35 efried In fact, I think that it might "just work" after Sylvain's vgpu stuff lands to hot-(un)plug a GPU.
16:01:45 efried if you unplug one that has allocations against it, you can expect problems; but otherwise...
16:02:07 efried and even then, I think we've accounted for that by setting reserved=total.
16:02:37 sean-k-mooney well there were edgecases like this https://review.openstack.org/#/c/626381/
16:03:12 sean-k-mooney if the device you remove had allocation against it we cant remove it form placemetn until those allocation are also removed
16:03:48 sean-k-mooney so the auto updating is greate but there are also dragons lurking there
16:05:09 efried right, like I said, that case is accounted for (at least I definitely remember this code in the xen vgpu patches) where, if a card goes away, but has allocations against it, we set reserved=total so no futher allocations can be made against it, until all the existing allocations go away, at which point we can delete the provider.
16:06:12 openstackgerrit Balazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule https://review.openstack.org/619528
16:06:12 openstackgerrit Balazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping https://review.openstack.org/616239
16:06:12 sean-k-mooney ya so we have solution for this edgecase and likely other but its a little more involved then you would assume at first glance
16:06:13 openstackgerrit Balazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule https://review.openstack.org/619529
16:06:13 openstackgerrit Balazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api https://review.openstack.org/616240
16:06:14 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317
16:06:14 openstackgerrit Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459
16:06:15 openstackgerrit Balazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest https://review.openstack.org/625310
16:06:15 openstackgerrit Balazs Gibizer proposed openstack/nova master: Remove port allocation during detach https://review.openstack.org/622421
16:06:16 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543
16:06:16 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist https://review.openstack.org/625311
16:06:17 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360
16:07:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360
16:09:39 mriedem lyarwood: mdbooth: +2 on https://review.openstack.org/#/c/551302/
16:09:43 mriedem dansmith: ^
16:11:24 mdbooth mriedem: Thanks!
16:15:00 lyarwood mriedem: cheers
16:16:03 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support server create with ports having resource request https://review.openstack.org/636360
16:17:20 efried aspiers, mriedem: Were either of you planning to resurrect https://review.openstack.org/#/c/538498/ ?
16:17:54 sean-k-mooney mriedem: when you have a sec can you let me know if the responces in https://review.openstack.org/#/c/624842/8//COMMIT_MSG make sense. if you are happy with them i can resping with that info added to the commit and your other comments addressed in an hour or so
16:18:21 mriedem efried: i'm not working on it
16:19:45 stephenfin efried: Comments on https://review.openstack.org/#/c/582392/
16:19:51 efried ...
16:19:52 mriedem sean-k-mooney: i guess....
16:20:05 mriedem setting the vif mac to 0s works depending on the underlying driver
16:20:08 mriedem is the summary i'm getting
16:20:15 sean-k-mooney yes
16:20:19 mriedem i'm not sure how the minimum libvirt version plays into any of that
16:20:37 sean-k-mooney oh with libvirt 3.2 it takes care of all of this
16:20:42 mriedem set_vf_interface_vlan doesn't document it's expected exit codes at all either so i had no idea
16:20:49 sean-k-mooney so we can delete this code
16:23:57 aspiers efried: I started to rebase it and got confused by the conflicts through lack of familiarity with the code, so I installed git-irebase and started to incrementally rebase it one commit at a time, so that the conflicts are smaller and there's more chance of me understanding how to handle them correctly
16:24:15 efried aspiers: tldr you're working on it?
16:24:23 aspiers well, trying ;-)
16:24:39 aspiers I reserve the right to give up through incompetence though
16:25:02 sean-k-mooney mriedem: the behavior chagne 2 weeks ago in https://github.com/openstack/nova/commit/13e283cc82e7cab564850501f9781385fb7b3f68 ill have to respin anyway now.
16:25:12 aspiers efried, mriedem: however I've just taken a brief pause to review the SEV patches my colleague just posted
16:25:54 efried aspiers: Just lmk if you do decide to give up on it, and I'll take over. I want it to get finished one way or another. (No particular hurry, just don't want it stalled indefinitely.)
16:25:57 mriedem sean-k-mooney: behavior change?
16:26:02 aspiers efried: sure thing
16:26:09 efried and thank you.
16:26:13 aspiers yw
16:27:08 mriedem efried: have you taken a gander at https://review.openstack.org/#/c/616239/ yet? that's gibi's "Calculate RequestGroup resource provider mapping" change which i figure you're the best person to review that
16:27:12 sean-k-mooney mriedem: actully no never mind the exit_code = [0, 2, 254] is now donw internally in another fucntion instead of pased in

Earlier   Later