| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-02 | |||
| 19:05:17 | efried | And does a spawn | |
| 19:05:49 | efried | Most of nova doesn't notice the GPU resource in the extra specs, just ignores it. But once we get into the powervm virt driver, we see it and act on it. | |
| 19:06:09 | efried | We see it in the allocation, to be clear. We don't care so much about the flavor at this point. | |
| 19:06:44 | efried | In the allocation we can see which provider UUID the resource came from. And we know which actual device that corresponds to on the system. (Because we cleverly named the RP after the DRC index, or something.) | |
| 19:07:08 | efried | We create the VM as normal, but before boot, we have to attach the device. | |
| 19:07:52 | efried | We do that by calling another REST API, probably POST /rest/api/uom/LogicalPartition/{uuid} with a payload including the <IOSlot/> entry corresponding to the device corresponding to the resource provider we peeled out of the allocation. | |
| 19:08:18 | efried | That REST API does magic on the platform to attach the device to the VM. Then we boot, and we're done. | |
| 19:08:21 | efried | So | |
| 19:08:24 | efried | In cyborg-land | |
| 19:08:49 | efried | that POST /rest/api/uom/LogicalPartition/{uuid} thing would be the guts of the os-acc plugin's plug() operation. | |
| 19:08:55 | efried | easy peasy. | |
| 19:09:00 | efried | But what about the discovery part? | |
| 19:10:04 | efried | This is where Cyborg shouldn't have to know anything about GET /rest/api/uom/ManagedSystem and how to parse the XML payload that comes in response. | |
| 19:10:16 | efried | or even the much easier pypowervm equivalent of ^ | |
| 19:10:35 | Sundar | Cool. The core of this flow -- representation in terms of RPs/RCs/traits and making Nova agnostic about the GPU resource class -- are the same as Cyborg ... | |
| 19:11:21 | efried | There's a reason for that :) | |
| 19:11:21 | Sundar | A few questions: who runs the discovery API (GET of /rest/api/uom/ManagedSystem)? Is it something on the compute node? | |
| 19:12:08 | efried | Yes, in pre-cyborg-land, the discovery happens in the powervm virt driver code, during get_available_resource / update_provider_tree | |
| 19:13:33 | efried | Note that this flow is in the early stages of development right now. We expect to have it working in our out-of-tree driver in Rocky. What we have currently implemented is for SR-IOV and is very different. | |
| 19:13:59 | Sundar | OK, for the other REST API (POST /rest/api/uom/LogicalPartition/{uuid}), something about the implementation has to come back to the compute node and poke around the device, right? If the pypowervm is doing that, why do you need the REST API? The pypowervm can just do the poking around. | |
| 19:14:06 | efried | The discovery and attach are similarly themed, in the sense that they involve PowerVM REST API calls. But we're not doing anything with resource providers yet. | |
| 19:14:32 | efried | "come back to the compute node and poke around the device"... | |
| 19:14:44 | efried | This is the thing about the power platform. | |
| 19:15:00 | efried | The management partition is not the hypervisor. | |
| 19:16:25 | efried | Power has true virtualization, in the sense that the host owns the hardware, and the "hypervisor" is a firmware thingy that I don't understand, and the management partition can only do its work by talking to that firmware gizmo. | |
| 19:17:04 | Sundar | So, the powervm virt driver is running in the mgmt partition and does not know about the hypervisor. Does the hypervisor run device drivers? | |
| 19:17:27 | efried | not in the sense that you think of device drivers. | |
| 19:18:24 | efried | One of the main points to note is that the devices themselves aren't "visible" to the management partition, unless you actually *attach* them to it. | |
| 19:18:53 | Sundar | How would a GPU device vendor support Power? Does he have to write a plugin/whatever for the hypervisor? | |
| 19:18:55 | efried | This is why pypowervm can't just "poke around the device" directly. | |
| 19:19:22 | efried | Somebody has to write drivers, yeah. | |
| 19:19:36 | efried | You're way out of my area of expertise here, though. | |
| 19:20:18 | Sundar | Hmmm, I need to think about this. | |
| 19:20:37 | Sundar | Would SR-IOV device flow differ from the above in any significant way? | |
| 19:20:38 | efried | I have no idea how e.g. FPGA programming is going to work. I'm only guessing it'll work like I mentioned above, where we'll have to attach the device to the management partition, where we'll have the right device driver installed, and we'll do the programming there, and then detach from the mgmt partition and attach to the VM. | |
| 19:21:12 | efried | So SR-IOV we actually have working today, just without placement in the picture; it's all hacked into the old PCI passthrough subsystem. | |
| 19:21:35 | efried | But SR-IOV is special (even more so) on power. | |
| 19:22:28 | efried | We virtualize the VF through the virtial I/O servers. (That's three virtuals in that sentence.) This allows us to do redundancy at the platform level (rather than aggregation at the VM) and it also enables live migration of SR-IOV VMs. | |
| 19:23:07 | efried | But - the discovery and plug steps are much the same in the sense that we do them by making REST calls. | |
| 19:23:34 | efried | It's just the nova flow that's totally different at the moment. I'm actually not sure how we're going to do SR-IOV with placement. Maybe just not, yet. | |
| 19:23:45 | efried | edmondsw: Have we considered ^ ? | |
| 19:24:22 | Sundar | Power's mgmt vs. hypervisor split sounds a bit like Xen's Dom 0 vs hypervisor split, except Xen drivers in dom0 can access devices directly (no REST API needed). Does that sound about right? | |
| 19:26:14 | efried | I'm afraid I don't know. | |
| 19:27:59 | Sundar | OK, let me process that a bit. Thanks, efried. I didn't think I would get a crash course in Power as part of Cyborg development. ;) | |
| 19:28:15 | efried | Sundar: Nobody expects the Spanish Inquisition. | |
| 19:28:35 | efried | And I'll thank you not to use "crash" and "Power" in the same sentence again. | |
| 19:28:41 | efried | Never happens. | |
| 19:28:54 | Sundar | haha, sure. | |
| 19:29:23 | efried | Thanks for the chat, Sundar. Keep up the good work, sir. | |
| 19:29:54 | Sundar | NP. Take care, efried. | |
| 19:56:10 | edmondsw | efried that's long... what were you asking? | |
| 19:56:59 | efried | edmondsw: What, if anything, our plan is for porting our SR-IOV support over to placement-land | |
| 19:57:55 | edmondsw | efried I thought you were considering that, but I have not spent time on it | |
| 19:59:02 | efried | edmondsw: I haven't really considered it yet, no. I think for the forseeable future if we continue to expose SR-IOV devices via PCI passthrough it will *probably* just work with the GPU-via-placement stuff. | |
| 19:59:13 | efried | But eventually we'll want to get it all into placement-land. | |
| 19:59:22 | efried | I just don't know what that's going to look like. | |
| 20:00:00 | edmondsw | I must be missing something. Hasn't SR-IOV been one of the example use cases in several conversations? | |
| 20:00:20 | efried | example use case of what, in conversations about what? | |
| 20:00:38 | edmondsw | placement, traits, resource providers, etc. | |
| 20:00:44 | edmondsw | nrp | |
| 20:00:49 | efried | I've talked a lot about SR-IOV in general, hypothetical terms when discussing those things, yes. | |
| 20:01:05 | efried | But we haven't ever gone into the specifics. | |
| 20:04:08 | openstackgerrit | Merged openstack/nova master: Request only instance_uuid in ironic node list https://review.openstack.org/539509 | |
| 20:05:53 | openstackgerrit | Matt Rabe proposed openstack/nova master: Add destination MSP IP address to PowerVM migrate data https://review.openstack.org/579676 | |
| 21:27:08 | openstackgerrit | Matt Rabe proposed openstack/nova master: Add destination MSP IP address to PowerVM migrate data https://review.openstack.org/579676 | |
| 21:28:45 | openstack | bug 1779635 in OpenStack Compute (nova) "placement allows RP parent loop in PUT resource_providers/{uuid}" [Medium,In progress] https://launchpad.net/bugs/1779635 - Assigned to Balazs Gibizer (balazs-gibizer) | |
| 21:28:45 | openstackgerrit | Merged openstack/nova master: Regression test for bug 1779635 https://review.openstack.org/579508 | |
| 23:11:10 | openstackgerrit | Merged openstack/nova master: Remove mox in unit/network/test_neutronv2.py (1) https://review.openstack.org/572018 | |
| #openstack-nova - 2018-07-03 | |||
| 01:08:11 | naichuans_ | efried: Thank you, will wait until reshape finished :) | |
| 01:31:28 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Add support specify volume type when boot instance https://review.openstack.org/579520 | |
| 01:49:49 | gmann | deepak_mourya: i think natsume replied on bug. 'id' is in ignored sort_keys - https://github.com/openstack/nova/blob/9382b7854286bba21235c4bfe39c585a6b894419/nova/api/openstack/compute/schemas/servers.py#L429 | |
| 02:38:27 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add policy to InstanceGroup object and api models. https://review.openstack.org/563375 | |
| 03:06:19 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add policy field to ServerGroup notification object https://review.openstack.org/563401 | |
| 03:12:57 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add fault info when unshelve instance failed https://review.openstack.org/579747 | |
| 03:31:55 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Change the anti-affinity Filter to adapt to new policy https://review.openstack.org/571166 | |
| 03:32:06 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Change the anti-affinity Filter to adapt to new policy https://review.openstack.org/571166 | |
| 03:32:38 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Adapt _validate_instance_group_policy to new policy model https://review.openstack.org/571465 | |
| 03:33:04 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Microversion 2.64 - Use new format policy in server group https://review.openstack.org/567534 | |
| 03:56:45 | openstackgerrit | Merged openstack/osc-placement master: Add nested resource providers (v1.14) https://review.openstack.org/546675 | |
| 03:56:46 | openstackgerrit | Merged openstack/osc-placement master: Limit allocation candidates (v1.15, v1.16) https://review.openstack.org/548043 | |
| 03:56:47 | openstackgerrit | Merged openstack/osc-placement master: Allocation candidates parameter: required (v1.17) https://review.openstack.org/548326 | |
| 04:36:33 | openstackgerrit | Deepak Mourya proposed openstack/nova master: Resource_provider API handler does not return specific error codes https://review.openstack.org/579423 | |
| 06:54:42 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/578019 | |
| 07:31:38 | openstackgerrit | Tuan Do Anh proposed openstack/nova-specs master: fix tox python3 overrides https://review.openstack.org/579793 | |
| 07:47:00 | openstackgerrit | Lei Zhang proposed openstack/nova master: Add method to get cpu traits https://review.openstack.org/560317 | |
| 07:56:54 | openstackgerrit | Claudiu Belu proposed openstack/nova master: tests: autospecs all the mock.patch usages https://review.openstack.org/470775 | |
| 07:56:55 | openstackgerrit | Claudiu Belu proposed openstack/nova master: hyper-v: autospec classes before they are instantiated https://review.openstack.org/342211 | |
| 08:01:02 | openstackgerrit | Claudiu Belu proposed openstack/nova master: WIP: replace spec with autospec https://review.openstack.org/557299 | |
| 08:08:28 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Prevent updating an RP's parent to form a loop https://review.openstack.org/579618 | |
| 08:31:28 | openstackgerrit | Merged openstack/nova master: Merge server create schema for keypair extension https://review.openstack.org/578332 | |
| 08:33:54 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add InstanceGroupPolicy object https://review.openstack.org/573628 | |
| 08:33:55 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Refactor the policies to policy https://review.openstack.org/579113 | |
| 08:33:56 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add policy to InstanceGroup object and api models. https://review.openstack.org/563375 | |
| 08:33:57 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add policy field to ServerGroup notification object https://review.openstack.org/563401 | |
| 08:33:58 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Change the anti-affinity Filter to adapt to new policy https://review.openstack.org/571166 | |
| 08:33:59 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Adapt _validate_instance_group_policy to new policy model https://review.openstack.org/571465 | |
| 08:34:00 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Microversion 2.64 - Use new format policy in server group https://review.openstack.org/567534 | |
| 08:50:27 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform missing delete notifications https://review.openstack.org/410297 | |
| 09:06:17 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Transform instance-live_migration_post notification https://review.openstack.org/480119 | |
| 09:10:29 | openstackgerrit | jiang wei proposed openstack/nova master: Add action initiator attribute to the instance payload https://review.openstack.org/536243 | |