| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-04 | |||
| 13:01:32 | gmann | takashin: i feel we should return 'type' always | |
| 13:01:53 | takashin | gmann: Thank you for your review. | |
| 13:02:08 | takashin | gmann: I will check your comment. | |
| 13:02:30 | gmann | takashin: thanks. i am reviewing tempest patch also, hope we get all these in soon. | |
| 13:03:15 | gmann | takashin: got that while checking the schema where you added 'type' as required param | |
| 13:09:54 | jaypipes | bauzas: sorry, reading back... electricians at my house | |
| 13:11:53 | mriedem | still looking for a final +2 on this pike regression fix https://review.openstack.org/#/c/507938/ | |
| 13:12:05 | mriedem | bauzas: ^ is in your wheelhouse | |
| 13:12:52 | bauzas | mriedem: oh coolness | |
| 13:14:24 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Moving more utils to ServerResourceAllocationTestBase https://review.openstack.org/499539 | |
| 13:14:25 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: factor out compute service start in ServerMovingTest https://review.openstack.org/503037 | |
| 13:14:25 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test resource allocation during soft delete https://review.openstack.org/495159 | |
| 13:15:26 | bauzas | mriedem: if you remember, I also have https://review.openstack.org/#/c/481116/ that is related | |
| 13:15:36 | bauzas | I need to rebase it | |
| 13:17:09 | bauzas | jaypipes: np, my 2nd question is more about how you see the virt driver consuming that specific allocation | |
| 13:17:25 | bauzas | jaypipes: that will be set by the scheduler | |
| 13:18:02 | bauzas | jaypipes: for the moment, we can just do things in the virt driver by introspecting the flavor and see that if we have VGPU:1 in the flavor, we need to hook up a mdev device | |
| 13:18:22 | bauzas | jaypipes: but that's not a super clear interface | |
| 13:18:40 | bauzas | note that the problem is identical for Xen | |
| 13:18:50 | bauzas | with the slight detail it doesn't use mdevs, of course | |
| 13:19:27 | sahid | bauzas, jaypipes: without talking about mdev, how that is going to work for the physical devices, which is on what you are working i think | |
| 13:19:52 | mriedem | maciejjozefczyk: question about https://review.openstack.org/#/c/494973/ - it was listed as Partial-Bug fix in the commit message but why? is there more to fix in that bug? | |
| 13:19:59 | jaypipes | bauzas: k, so the way we've been talking about that is that on startup, the generic device manager (or virt driver) would go through the host devices it discovers and populate a ProviderTree object supplied to it by the RT. The ProviderTree allows looking up providers by UUID or by name. So, when creating nodes in the ProviderTree, the device manager / virt driver would create the resource provider with a UUID and a unique name. It would then | |
| 13:19:59 | jaypipes | look up resource providers by UUID or by name later on | |
| 13:20:54 | bauzas | jaypipes: that I understood | |
| 13:21:10 | bauzas | jaypipes: how we consume that ? by passing the allocation down to the compute ? | |
| 13:22:35 | jaypipes | bauzas: yes, the allocation request contains the resource provider UUID(s) of the providers providing resources for an instance. If the scheduler claimed one VGPU resource against a particular device represented by UUID1, the allocation request will contain UUID1: {resources: {VGPU: 1}}} | |
| 13:23:19 | jaypipes | bauzas: and it will be up to the virt driver or generic device manager to look up which device corresponds to which UUID. | |
| 13:23:25 | gmann | sdague: mriedem need your eyes and feedback on policy removal list in this spec - https://review.openstack.org/#/c/508101/ | |
| 13:23:31 | bauzas | jaypipes: so that requires https://review.openstack.org/#/c/486215 | |
| 13:24:17 | bauzas | jaypipes: if we say we're just going to do a quick POC for Queens with vGPUs, we won't have that yet, so permission to just introspect the flavor in the virt driver ? | |
| 13:24:19 | jaypipes | bauzas: not really, no. The compute host looks up allocations for an instance at the moment by doing a GET /allocations/{consumer_uuid} in the RT | |
| 13:24:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance. |
|
| 13:24:22 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance. |
|
| 13:24:22 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: use already loaded BDM in instance.create https://review.openstack.org/483969 | |
| 13:24:28 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: doc: fix flavor notes https://review.openstack.org/509438 | |
| 13:24:29 | jaypipes | bauzas: the alternate hosts thing is for retry ability. | |
| 13:24:32 | bauzas | jaypipes: oh that's right | |
| 13:24:38 | bauzas | we don't need to pass them over RPC | |
| 13:24:59 | bauzas | because compute is already looking up them | |
| 13:25:22 | bauzas | okay, so the RT knows the allocation, it can then passes the allocation to the virt driver, right? | |
| 13:25:31 | bauzas | jaypipes: ^ | |
| 13:25:34 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509439 | |
| 13:25:59 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509440 | |
| 13:26:02 | jaypipes | bauzas: yes. | |
| 13:26:09 | bauzas | gotcha | |
| 13:26:11 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/newton: Account for compute.metrics.update in legacy notification whitelist https://review.openstack.org/509441 | |
| 13:26:46 | bauzas | jaypipes: sahid: I think it's enough for us to be able to assign vGPUs for Queens | |
| 13:27:09 | mriedem | sdague: how do you feel about this backport https://review.openstack.org/#/c/505546/ ? | |
| 13:27:12 | jaypipes | bauzas, sahid: if the virt driver (or generic device manager in the future) wants to store that mapping of mdev identifier to resource provider UUID in a DB table (pci_devices?), cool. If it wants to store it in etcd, cool. An inventory.yaml file on the host? also cool, doesn't matter to me :) | |
| 13:27:24 | bauzas | sahid: the virt driver reports how many vGPUs it can assign by filling in the get_inventory() method | |
| 13:27:40 | mriedem | sdague: i think it's ok, it's adding the ability to specify certs when talking to keystone for the os-quota-sets and flavor-access APIs | |
| 13:28:17 | bauzas | sahid: then at the creation time, the RT would pass the allocated claim (I mean the allocation record) to the virt driver so it would get the RP UUID and the amount to consume | |
| 13:28:31 | sahid | jaypipes: it' good point, since currently libvirt is using PciDevice, so it will be a good transition then | |
| 13:28:37 | bauzas | then the virt driver would allocate from the pool of physical devices it manages | |
| 13:28:55 | jaypipes | sahid: yup | |
| 13:29:40 | jaypipes | bauzas: no, not really... the device will have already been picked by the scheduler. all the virt driver would need to do is plumb the specific device to the guest. | |
| 13:29:53 | bauzas | jaypipes: sahid: I guess the most important matter is that whatever the technical persistence is, it's not provided outside of the virt driver | |
| 13:30:03 | jaypipes | bauzas: and that's the point we're trying to get to. the RT and scheduler do the claiming/allocating resources stuff and the virt driver does the guest plumbing. | |
| 13:30:23 | bauzas | jaypipes: if we pass VGPU:8 as a RC | |
| 13:30:41 | bauzas | jaypipes: then the allocation would be against the root RP | |
| 13:30:45 | bauzas | for queens I mean | |
| 13:30:57 | jaypipes | bauzas: no, not necessarily. | |
| 13:31:08 | bauzas | I'm all ears :) | |
| 13:31:39 | jaypipes | bauzas: we're aiming to get n-r-p work done in Queens. so, the allocation would be against one or more child providers (in libvirt, those would be pGPUs, in Xen they would be pGPU groups). | |
| 13:32:53 | jaypipes | bauzas: so all the virt driver would be responsible for doing is a) looking up physical device information by resource provider UUID and b) doing the necessary guest plumbing for the device (in other words, in libvirt's case, writing the XML snippet information for the device, etc) | |
| 13:33:01 | bauzas | I agree, I just thought we said we could try to provide GPU resources as a global resource class for the node in Queens | |
| 13:33:41 | bauzas | if nested-RPs is already there, then of course we would modify that to just lookup the child RP | |
| 13:34:13 | dansmith | bauzas: yes that's wht we should do | |
| 13:34:27 | dansmith | bauzas: we can expose gpu resources for the compute node right now | |
| 13:36:26 | sahid | efried: please ping me when you have a moment so we can talk about your work on the generic devices management | |
| 13:36:29 | efried | jaypipes GDM dig accepted | |
| 13:36:40 | jaypipes | efried: lol :) | |
| 13:36:42 | efried | sahid Now's good, unless I need to catch up on the ML first. | |
| 13:37:18 | efried | jaypipes BUT - I've actually been thinking along the lines that, once NRP is in place, there will be no need for such a thing as a GDM. | |
| 13:37:35 | sahid | i did not expected to see you respond so quickly :) | |
| 13:37:48 | maciejjozefczyk | mriedem: Yes other patch should be applied to rollback migration when delete is called, similiar to: https://review.openstack.org/#/c/185958/ | |
| 13:38:16 | maciejjozefczyk | mriedem: I solved an effect of broken migration, not the source | |
| 13:38:29 | jaypipes | efried: oh, there still will be. | |
| 13:38:42 | jaypipes | efried: there's still a need for discovery of hardware on the host. | |
| 13:38:48 | efried | virt driver | |
| 13:38:58 | sahid | efried: do you have some pointers of work in progress? | |
| 13:39:00 | efried | whitelisting? virt driver | |
| 13:39:04 | jaypipes | efried: doing so would probably lead to a lot of dup code. | |
| 13:39:17 | efried | Mapping devices to RPs? virt driver. | |
| 13:39:49 | efried | sahid The only "work in progress" is scribbles on etherpads, which we discussed at the PTG. | |
| 13:40:09 | sahid | efried: yes i was not here, if you can give me the link | |
| 13:40:32 | efried | sahid https://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management | |
| 13:40:49 | efried | sahid It has generally been a drive towards understanding how devices are going to be managed once we go full-bore with placement & resource providers | |
| 13:40:59 | efried | sahid The goal being to get rid of the existing PCI manager code. | |
| 13:41:25 | mriedem | gmann_sleep: doesn't it seem odd that we still have this filtering code when listing instances to be able to filter by metadata and system_metadata? https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2328-L2332 | |
| 13:41:32 | mriedem | i thought that was a 400 in the API now | |
| 13:41:56 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L182 | |
| 13:42:14 | efried | jaypipes Potentially duplication among the linuxy hypervisors, I suppose. I would still expect the code to ultimately run under the auspices of the virt driver. | |
| 13:42:15 | sahid | efried: no transition phase as suggested jaypipes by adding a update_from_inventory() method to the PciManager? | |
| 13:43:15 | efried | jaypipes So I could see having some shared class above the ComputeDriver base class that provides linuxy impls for devicey methods. | |
| 13:44:15 | efried | sahid Well, the cores have made their position pretty clear: Investment in the existing PCI manager is going to be very limited. | |
| 13:46:19 | efried | sahid That said, I think it may be possible to get very close to what you want just using NRP with traits and careful modeling, which is happening in Queens. | |
| 13:46:44 | efried | sahid Fancy use cases like (anti)affinity won't work yet. | |
| 13:46:49 | jaypipes | efried: yeah, we're gonna need to have some sort of transitionary plan anyway... | |