| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-04 | |||
| 13:24:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance. |
|
| 13:24:22 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance.create https://review.openstack.org/483969 | |
| 13:24:22 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance. |
|
| 13:24:28 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: doc: fix flavor notes https://review.openstack.org/509438 | |
| 13:24:29 | jaypipes | bauzas: the alternate hosts thing is for retry ability. | |
| 13:24:32 | bauzas | jaypipes: oh that's right | |
| 13:24:38 | bauzas | we don't need to pass them over RPC | |
| 13:24:59 | bauzas | because compute is already looking up them | |
| 13:25:22 | bauzas | okay, so the RT knows the allocation, it can then passes the allocation to the virt driver, right? | |
| 13:25:31 | bauzas | jaypipes: ^ | |
| 13:25:34 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509439 | |
| 13:25:59 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509440 | |
| 13:26:02 | jaypipes | bauzas: yes. | |
| 13:26:09 | bauzas | gotcha | |
| 13:26:11 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/newton: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509441 | |
| 13:26:46 | bauzas | jaypipes: sahid: I think it's enough for us to be able to assign vGPUs for Queens | |
| 13:27:09 | mriedem | sdague: how do you feel about this backport https://review.openstack.org/#/c/505546/ ? | |
| 13:27:12 | jaypipes | bauzas, sahid: if the virt driver (or generic device manager in the future) wants to store that mapping of mdev identifier to resource provider UUID in a DB table (pci_devices?), cool. If it wants to store it in etcd, cool. An inventory.yaml file on the host? also cool, doesn't matter to me :) | |
| 13:27:24 | bauzas | sahid: the virt driver reports how many vGPUs it can assign by filling in the get_inventory() method | |
| 13:27:40 | mriedem | sdague: i think it's ok, it's adding the ability to specify certs when talking to keystone for the os-quota-sets and flavor-access APIs | |
| 13:28:17 | bauzas | sahid: then at the creation time, the RT would pass the allocated claim (I mean the allocation record) to the virt driver so it would get the RP UUID and the amount to consume | |
| 13:28:31 | sahid | jaypipes: it' good point, since currently libvirt is using PciDevice, so it will be a good transition then | |
| 13:28:37 | bauzas | then the virt driver would allocate from the pool of physical devices it manages | |
| 13:28:55 | jaypipes | sahid: yup | |
| 13:29:40 | jaypipes | bauzas: no, not really... the device will have already been picked by the scheduler. all the virt driver would need to do is plumb the specific device to the guest. | |
| 13:29:53 | bauzas | jaypipes: sahid: I guess the most important matter is that whatever the technical persistence is, it's not provided outside of the virt driver | |
| 13:30:03 | jaypipes | bauzas: and that's the point we're trying to get to. the RT and scheduler do the claiming/allocating resources stuff and the virt driver does the guest plumbing. | |
| 13:30:23 | bauzas | jaypipes: if we pass VGPU:8 as a RC | |
| 13:30:41 | bauzas | jaypipes: then the allocation would be against the root RP | |
| 13:30:45 | bauzas | for queens I mean | |
| 13:30:57 | jaypipes | bauzas: no, not necessarily. | |
| 13:31:08 | bauzas | I'm all ears :) | |
| 13:31:39 | jaypipes | bauzas: we're aiming to get n-r-p work done in Queens. so, the allocation would be against one or more child providers (in libvirt, those would be pGPUs, in Xen they would be pGPU groups). | |
| 13:32:53 | jaypipes | bauzas: so all the virt driver would be responsible for doing is a) looking up physical device information by resource provider UUID and b) doing the necessary guest plumbing for the device (in other words, in libvirt's case, writing the XML snippet information for the device, etc) | |
| 13:33:01 | bauzas | I agree, I just thought we said we could try to provide GPU resources as a global resource class for the node in Queens | |
| 13:33:41 | bauzas | if nested-RPs is already there, then of course we would modify that to just lookup the child RP | |
| 13:34:13 | dansmith | bauzas: yes that's wht we should do | |
| 13:34:27 | dansmith | bauzas: we can expose gpu resources for the compute node right now | |
| 13:36:26 | sahid | efried: please ping me when you have a moment so we can talk about your work on the generic devices management | |
| 13:36:29 | efried | jaypipes GDM dig accepted | |
| 13:36:40 | jaypipes | efried: lol :) | |
| 13:36:42 | efried | sahid Now's good, unless I need to catch up on the ML first. | |
| 13:37:18 | efried | jaypipes BUT - I've actually been thinking along the lines that, once NRP is in place, there will be no need for such a thing as a GDM. | |
| 13:37:35 | sahid | i did not expected to see you respond so quickly :) | |
| 13:37:48 | maciejjozefczyk | mriedem: Yes other patch should be applied to rollback migration when delete is called, similiar to: https://review.openstack.org/#/c/185958/ | |
| 13:38:16 | maciejjozefczyk | mriedem: I solved an effect of broken migration, not the source | |
| 13:38:29 | jaypipes | efried: oh, there still will be. | |
| 13:38:42 | jaypipes | efried: there's still a need for discovery of hardware on the host. | |
| 13:38:48 | efried | virt driver | |
| 13:38:58 | sahid | efried: do you have some pointers of work in progress? | |
| 13:39:00 | efried | whitelisting? virt driver | |
| 13:39:04 | jaypipes | efried: doing so would probably lead to a lot of dup code. | |
| 13:39:17 | efried | Mapping devices to RPs? virt driver. | |
| 13:39:49 | efried | sahid The only "work in progress" is scribbles on etherpads, which we discussed at the PTG. | |
| 13:40:09 | sahid | efried: yes i was not here, if you can give me the link | |
| 13:40:32 | efried | sahid https://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management | |
| 13:40:49 | efried | sahid It has generally been a drive towards understanding how devices are going to be managed once we go full-bore with placement & resource providers | |
| 13:40:59 | efried | sahid The goal being to get rid of the existing PCI manager code. | |
| 13:41:25 | mriedem | gmann_sleep: doesn't it seem odd that we still have this filtering code when listing instances to be able to filter by metadata and system_metadata? https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2328-L2332 | |
| 13:41:32 | mriedem | i thought that was a 400 in the API now | |
| 13:41:56 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L182 | |
| 13:42:14 | efried | jaypipes Potentially duplication among the linuxy hypervisors, I suppose. I would still expect the code to ultimately run under the auspices of the virt driver. | |
| 13:42:15 | sahid | efried: no transition phase as suggested jaypipes by adding a update_from_inventory() method to the PciManager? | |
| 13:43:15 | efried | jaypipes So I could see having some shared class above the ComputeDriver base class that provides linuxy impls for devicey methods. | |
| 13:44:15 | efried | sahid Well, the cores have made their position pretty clear: Investment in the existing PCI manager is going to be very limited. | |
| 13:46:19 | efried | sahid That said, I think it may be possible to get very close to what you want just using NRP with traits and careful modeling, which is happening in Queens. | |
| 13:46:44 | efried | sahid Fancy use cases like (anti)affinity won't work yet. | |
| 13:46:49 | jaypipes | efried: yeah, we're gonna need to have some sort of transitionary plan anyway... | |
| 13:47:33 | efried | jaypipes Oh, like sahid is saying with update_from_inventory() - presumably a RT method that peels dev info out of the virt inventory and flushes it back to the PCI manager? | |
| 13:47:54 | jaypipes | efried: chatting with dansmith and bauzas, I'm cool with trying to get single-inventory VGPU support pushed for Queens if we miss nested resource provider targets. That would means no support for multiple GPU types on a single compute node, but would at least get some rudimentary VGPU support | |
| 13:49:08 | dansmith | jaypipes: multiple gpu types can be handled with aggregates until we have richer support, and at least one customer has told me that's fine, FWIW | |
| 13:49:18 | bauzas | efried: I just wanted to clarify how we would translate a specific {'VPGU:1"} request into something very virt-specific | |
| 13:49:18 | jaypipes | k | |
| 13:49:40 | bauzas | heh VGPUs even | |
| 13:49:41 | sahid | it seems that XenServer provides an abstraction which makes easy what you want to achieve | |
| 13:49:48 | sahid | but that does not look reasonable for libvirt | |
| 13:50:05 | efried | bauzas So the only way we're going to translate an allocation to a specific device is if the RP is modeled as representing that specific device. | |
| 13:50:31 | sahid | XenServer is doing the managmenet ofr the devices, libvirt is using the PciManager | |
| 13:50:56 | bauzas | efried: sahid: dansmith: I just feel we need code so we could chat on the details | |
| 13:51:09 | bauzas | reporting the inventory for those resource classes is easy | |
| 13:51:21 | efried | The virt driver will ultimately be responsible for designing that model, maintaining that mapping, and providing the appropriate RPs and inventory to placement (via get_inventory, or update_provider_tree, or whatever it winds up being) | |
| 13:51:25 | dansmith | bauzas: you're asking how the virt driver knows a thing has been requested? | |
| 13:51:44 | bauzas | dansmith: yup, correct | |
| 13:51:57 | dansmith | bauzas: in the simple case it can just look at extra_specs on instance.flavor | |
| 13:52:00 | bauzas | dansmith: jaypipes told me RT looks up the allocation | |
| 13:52:11 | bauzas | dansmith: yeah that was my initial approach | |
| 13:52:17 | dansmith | bauzas: we have a routine that can parse the flavor and give you a merged resource view | |
| 13:52:27 | bauzas | that would be an easy thing then | |
| 13:52:33 | bauzas | okay, I need coding then | |
| 13:52:34 | dansmith | or look at the allocation, yeah, but plumbing that from rt to virt will take a little work | |
| 13:52:49 | efried | The virt driver will be able to see the allocation, no? | |
| 13:52:51 | bauzas | yeah, for a POC, introspecting the flavor seems the quickiest path | |
| 13:52:55 | dansmith | you will need the allocation once we have n-r-p or traits though | |
| 13:53:02 | bauzas | efried: it requires a new interface which we don't have yet | |
| 13:53:02 | mriedem | these -1s from zuul messing up my dashboard is messing up my life | |
| 13:53:10 | dansmith | efried: it could fetch it itself, yeah, but better if it didn't I think | |
| 13:53:30 | bauzas | dansmith: yeah, I'm not a fan of the virt driver calling placement | |
| 13:53:36 | dansmith | bauzas: yeah, a good first step would be figuring out the best way to tell the virt driver about the allocation | |
| 13:53:45 | dansmith | bauzas: yep | |
| 13:53:52 | efried | Talking like something in the Instance object? | |