| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-11 | |||
| 13:46:45 | efried | this takes hypervisor-specific code out of the RT, which is ++ | |
| 13:46:49 | efried | and fixes the race | |
| 13:47:01 | efried | and is reusable for other things like vpmem | |
| 13:47:21 | bauzas | efried: FWIW, this is somehow related to the spec you love :) | |
| 13:47:36 | efried | which, vgpu affinity? | |
| 13:48:21 | bauzas | efried: correct, as we check things in instance_claim() already | |
| 13:48:35 | bauzas | efried: and that's what I really want to amend for vGPUs | |
| 13:48:48 | efried | yeah, basically "figure out which specific thingy on the system is going to be assigned to the instance for a particular unit of resource class in an allocation" should be the purview of the virt driver. | |
| 13:49:00 | bauzas | as I said, the vGPU weigher and the object update aren't necessary | |
| 13:49:01 | efried | because only the virt driver has a view to "specific thingy on the system" | |
| 13:49:13 | bauzas | sec, giving you a link | |
| 13:50:54 | sean-k-mooney | bauzas: its even more related to the draft i wrote for that | |
| 13:52:39 | sean-k-mooney | efried: i think bauzas means https://review.opendev.org/#/c/650963/9/specs/train/approved/libvirt-vgpu-numa-affinity.rst | |
| 13:53:20 | bauzas | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2223 eventualls calls https://github.com/openstack/nova/blob/master/nova/compute/claims.py#L220 | |
| 13:53:27 | bauzas | efried: ^ | |
| 13:55:37 | bauzas | efried: so what you'd like is that the device assignment (ie. the mdev choice of https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6416) would be called by something close to https://github.com/openstack/nova/blob/master/nova/compute/claims.py#L220 ? | |
| 13:55:47 | bauzas | if so, I'm not opposed to | |
| 13:56:18 | bauzas | but xen needs substantial changes too | |
| 13:57:12 | coreycb | kashyap: cpaelzer added it to his 20.04 virt stack work. thanks for the heads up. | |
| 13:57:24 | efried | bauzas: I would like it if the hypervisor-specific aspects of all of that were in the virt driver -- but yes, to fix the race, the "mdev choice" needs to happen somewhere inside instance_claim. | |
| 13:57:35 | kashyap | coreycb: Cool. I'll send a note to the list as well once I get around | |
| 13:57:44 | coreycb | ok | |
| 13:57:45 | kashyap | Also need to remind the SUSE folks. And Debian | |
| 13:58:00 | efried | bauzas: I'll take this opportunity to point out that _test_numa_topology is far from hypervisor-agnostic. | |
| 13:58:04 | bauzas | efried: that's what I said yes | |
| 13:58:11 | sean-k-mooney | personally i think we should be calling calim_for_instacnce form the conductor by they way way before we get to the compute node after we select the host but before we call spawn | |
| 13:58:20 | bauzas | oh | |
| 13:58:22 | bauzas | surze | |
| 13:58:29 | bauzas | but that's a waaaay more complicated | |
| 13:58:34 | efried | baby steps | |
| 13:58:51 | sean-k-mooney | efried: babysteps that predate placement | |
| 13:58:59 | sean-k-mooney | we keep putting that off | |
| 13:59:36 | sean-k-mooney | it prevents all the races that placemetn will eventurally solve | |
| 14:00:20 | sean-k-mooney | if we do claim for instance in teh conductor via an rpc to the compute we nolonger race on any pci or numa resouce | |
| 14:00:31 | mriedem | meeting? | |
| 14:00:48 | artom | meating. | |
| 14:00:57 | efried | oh, craaap | |
| 14:01:25 | alex_xu | ok, that is my fault :) | |
| 14:11:38 | mnaser | i'm really curious if there's a possible case where nova-compute gets a request an complains abut InstanceActionNotFound_Remote | |
| 14:13:10 | mnaser | it means that the instane action was *not* created (or committed to db) when the rpc request was recieved ? | |
| 14:27:58 | mriedem | check the request id - the actions are keyed by request id so if the request id isn't the same for some reason it wouldn't be found | |
| 14:28:27 | mriedem | e.g. a periodic task in compute using an admin context with a generated request id hitting some method that expects an action to exist | |
| 14:46:52 | bauzas | efried: FWIW https://review.opendev.org/#/c/552924/ | |
| 14:47:13 | bauzas | it needs a bit of caring ^ | |
| 14:47:18 | efried | ah, nice bauzas, thanks. | |
| 14:52:04 | bauzas | mriedem: https://review.opendev.org/#/c/645520/35/doc/api_samples/servers/v2.74/server-create-req-with-only-host.json I thought we were no longer accepting to only provide a host field without also providing the node name ? | |
| 14:52:12 | bauzas | lemme look at the spec | |
| 14:52:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Convert nova-next to a zuul v3 job https://review.opendev.org/670196 | |
| 14:53:10 | bauzas | mriedem: hell, yeah, I already had that concern https://review.opendev.org/#/c/645458/18/specs/train/approved/add-host-and-hypervisor-hostname-flag-to-create-server.rst@125 | |
| 14:54:16 | mriedem | bauzas: not sure what you're talking about | |
| 14:54:31 | bauzas | mriedem: asking a destination by only the 'host' field | |
| 14:55:07 | mriedem | so what? for non-ironic the host == hypervisor_hostname | |
| 14:55:12 | mriedem | so why require the user to specify both? | |
| 14:55:15 | mriedem | that's just shitty ux imo | |
| 14:55:37 | bauzas | mriedem: I'm all good with only providing the node name | |
| 14:55:54 | bauzas | what I'm concerned is by only asking for a service name | |
| 14:56:14 | bauzas | and depending on the virt driver, the service name can be different from the node name | |
| 14:56:26 | mriedem | only for ironic, right? | |
| 14:56:30 | bauzas | I don't recall which one specifically | |
| 14:57:22 | mriedem | if you're using kvm, requesting the server land on a particular host is the same as asking that it land on a given node since they are the same thing | |
| 14:57:41 | mriedem | if you're using ironic, you can ask that it land on a particular compute service host meaning it will use a particular ironic cluster, you don't care which node, | |
| 14:57:51 | mriedem | or you can ask for a node and we'll lookup the host | |
| 14:57:58 | mriedem | or you can ask for both if you know exactly where it goes | |
| 14:58:05 | mriedem | as you'd do today for ironic | |
| 14:58:09 | mriedem | with the forced stuff for the JsonFilter | |
| 14:58:12 | mriedem | query hint | |
| 14:59:16 | sean-k-mooney | bauzas: we still hit the schdler so if we just pass the service name the schduler could chose a host managed by that ironic compute service | |
| 14:59:52 | sean-k-mooney | and ya i know that the hyperviors hosts can move between teh compute service but its proably still makes sense | |
| 15:00:27 | mriedem | i really really don't want to encode/enforce some dumb "you have to specify both host and node because of ironic" thing in the api | |
| 15:00:31 | bauzas | I just feel we're regressing with https://review.opendev.org/#/c/645520/35/nova/scheduler/host_manager.py | |
| 15:00:42 | mriedem | how? | |
| 15:00:47 | sean-k-mooney | oh and mriedem already said that above :) | |
| 15:03:52 | kenperkins | can someone point me to where in the instance create codepath the password gets generated when not provided? | |
| 15:04:26 | sean-k-mooney | do we always generate one? | |
| 15:04:54 | sean-k-mooney | i assumed not | |
| 15:05:05 | kenperkins | if we don't where does it come from? | |
| 15:05:08 | bauzas | mriedem: actually, if we need to keep the same UX as it is now, nodename is optional | |
| 15:05:19 | kenperkins | maybe it's baked into the image? <thinking> | |
| 15:05:36 | sean-k-mooney | some images have it backed in yes | |
| 15:05:56 | sean-k-mooney | we may always generate one but i was not expecting us too | |
| 15:06:15 | artom | donnyd, so, just to be clear, http://project.fortnebula.com/ is your personal project with your personal hardware, and... you're the sole sysadmin for it? | |
| 15:07:10 | donnyd | yep | |
| 15:07:29 | kenperkins | sean-k-mooney https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L581 perhaps? | |
| 15:07:56 | kenperkins | then here i think https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L1206 | |
| 15:08:09 | artom | donnyd, I'm worried about gate resources with a 1 person bottleneck :) For your sanity as much as cloud stability | |
| 15:08:33 | donnyd | artom: I am also the sole funder, maintainer, electrician, and cooling engineer | |
| 15:08:36 | donnyd | LOL | |
| 15:08:46 | sean-k-mooney | kenperkins: https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.inject_password | |
| 15:09:04 | sean-k-mooney | kenperkins: by defualt we do not inject admin password in the vm wit libvirt | |
| 15:09:11 | artom | donnyd, the bus factor is *really* high with you :) | |
| 15:09:21 | kenperkins | where is the default libvirt config in code? :D | |
| 15:09:37 | donnyd | Well openstack isn't very hard, so maintenance is pretty low | |
| 15:09:42 | sean-k-mooney | and its a libvirt specific option so other virt dirvres proably dont do this | |
| 15:10:09 | sean-k-mooney | donnyd: artom just be aware that we cant have any voting jobs without at least 2 providers | |
| 15:10:17 | mriedem | bauzas: hypervisor_hostname *is* optional in the new microversion | |
| 15:10:20 | mriedem | as is host | |
| 15:10:21 | artom | donnyd, I'll trust you on that one - my experience has been with devstack mostly, and maintenance is definitely *not* low | |
| 15:10:28 | mriedem | same as it is with the zone:host:node hack | |
| 15:10:35 | mriedem | you can specify zone:host only, or zone::node only | |
| 15:10:38 | kenperkins | @sean-k-mooney thx for the pointer re: libvirt config | |