| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2020-02-27 | |||
| 03:26:32 | Sundar | shaohe_feng: Just change the title for your patch, and maybe make 'multinode' as the topic. We could have a patch series for multinode, including Sean's patches | |
| 03:27:02 | Sundar | You already have the right topic | |
| 03:27:03 | shaohe_feng | mircroversion can also be fix one issues. | |
| 03:27:37 | shaohe_feng | and more patch fix other issues. | |
| 03:27:43 | shaohe_feng | This is a right way. | |
| 03:28:34 | shaohe_feng | or it will be the same as we fix "multinode" issues in devstack. last half years, and no progress. | |
| 03:28:56 | shaohe_feng | s/last/take | |
| 03:33:30 | shaohe_feng | ping | |
| 03:34:04 | brinzhang_ | xinranwang, Sundar, Agree the microversion schema do in next release, and it better done in this release, later we(xinranwang) can talk how to do it easy. | |
| 03:34:06 | chenke | Agree, a patch does not need to be modified too much. we can make a series of patches to speed up review. | |
| 03:35:28 | Sundar | chenke, shaoe_feng: Sure, organize in a series of patches. A reviewer may want to see a placeholder patch for the next step before agreeing to previous step? | |
| 03:35:38 | Sundar | shaohe_feng: ^ | |
| 03:35:52 | chenke | Ye. Agree | |
| 03:36:37 | chenke | reviewer's suggestions, we should carefully consider and give a reasonable response. | |
| 03:37:17 | Sundar | I think we are all in the same page. shaohe_feng may feel that his patch has not merged in a long time. Let's review it quickly and help it get merged. | |
| 03:37:23 | Sundar | Ok, anything else? | |
| 03:38:23 | shaohe_feng | Add NOTE in patch or summarize the tasks lists in etherpad to follow, if we want to address more issues. | |
| 03:38:42 | shaohe_feng | s/follow/track | |
| 03:39:21 | Sundar | Yumeng: you seem to be adding mdev support to Cyborg. That is interesting | |
| 03:39:52 | Sundar | Tht will require changes to Nova patches too | |
| 03:40:11 | Sundar | I am only allowing 'PCI' in the current patches. | |
| 03:40:13 | Yumeng | yes, I found that controlpatch_id_type of nvidia GPU should be "MDEV" | |
| 03:41:01 | Yumeng | Sundar: I was thinking just now: attach_handle type should also be "MDEV". do you agree? | |
| 03:41:06 | shaohe_feng | now the intel and NVIDIA vGPU are MDEV devices. | |
| 03:41:10 | Sundar | Yumeng: What do you mean that you found that? If you are doing PCI passthrough of the physical function, it would be 'PCI'. The mdeiated device is a different way of attaching a device, different from PCI passthrough. | |
| 03:41:11 | shaohe_feng | Only AMD's are SRIOV. | |
| 03:42:03 | Sundar | shaohe_feng and all: We had already discussed that Cyborg intends to support only physical PF passthrough for GPUs for now. | |
| 03:42:12 | Yumeng | Thanks shaohe_feng for pointing this. | |
| 03:42:50 | shaohe_feng | IMHO, MDEV can exist with nova's together for a long time. like PCI devices | |
| 03:42:53 | Sundar | What prevents a Nvidia GPU's PF from being passed through to a VM, instead of mdev? | |
| 03:43:59 | shaohe_feng | we should not prevents it. | |
| 03:46:04 | Sundar | Yumeng: Re. attach_handle type should also be "MDEV" -- yes, in fact control path ID cannot be 'mdev', only the attach handle can be. Because mdev refers to how the device is attached to a VM. | |
| 03:46:31 | Sundar | The cpid is PCI even for a device that supports mdev. | |
| 03:46:49 | Sundar | Because that is the management interface, which needs to be a PCI interface | |
| 03:46:54 | Yumeng | Sundar: ops. emmmm. I was testing vGPU, and found that controlpatch_id_type-- "MDEV" cannot be reported to DB Shema. | |
| 03:47:40 | Yumeng | so I was thinking that was a bug of DB. | |
| 03:48:28 | Sundar | Yumeng and all: do we intend to support vGPUs? In previous discussions, we said that Cyborg is for offload i.e. GPGPU type use cases, not media or graphics. So, the most comon usage is to ssign an entire GPU (may be multiple GPUs) to the same VM. | |
| 03:48:36 | Yumeng | seems that was an intend, right? | |
| 03:48:36 | Sundar | *assign | |
| 03:50:18 | brinzhang_ | Sundar, I would like we Cyborgcan support vGPU, it's an intend | |
| 03:50:49 | shaohe_feng | But I know some Public cloud has support vGPU | |
| 03:51:06 | shaohe_feng | why not we support it? | |
| 03:51:18 | Sundar | brinzhang_: Ho do we distinguish Cyborg from Nova, if both can do all use cases for GPUs? | |
| 03:51:24 | Sundar | *How | |
| 03:52:42 | Sundar | Anyway, I am fine with vGPU support if you all want it. | |
| 03:53:06 | shaohe_feng | and the first propose for smart-nic generic solution in kernel by Redhat also make the device under mdev bus. | |
| 03:53:57 | brinzhang_ | Sundar, I think Cyborg can provide the vGPU architecture to Nova, so Nova can choose the vGPU to use | |
| 03:55:14 | chenke | I think supporting vgpu is not a bad thing for cyborg. So, I agree. | |
| 03:55:50 | Sundar | brinzhang_: I don't know what that means. Nova already has vGPU support without Cyborg. Cyborg can report VFs as attach handles (with SR-IOV) or mdevs as attach handles (with Nvidia GPUs). Both would work AFAICS. It is just that Nova and CYborg overlap in functionality. | |
| 03:56:18 | Sundar | As i said, I am fine with vGPU support in Cyborg. However, I'd rather close the current Nova patch series as is. I have added only 'PCI" as supported type in https://review.opendev.org/#/c/631245/56/nova/virt/libvirt/driver.py@5698 | |
| 03:56:45 | Sundar | One of you could add 'mdev' to that. | |
| 03:57:08 | shaohe_feng | We know k8s, CNI for network, CSI for storage, DPI for device/accelerators, let the right components do the right functions | |
| 03:57:36 | shaohe_feng | It said they want to add mdev in cyborg | |
| 03:58:25 | shaohe_feng | but cyborg's progress is slow, so that they can wait it and add it to nova. | |
| 03:59:21 | Yumeng | Sundar: I think the good side is for the Operators(users). They can manage FPGA and vGPU using just one component cyborg. configuring at cyborg.conf instead of vGPU at nova.conf and FPGA at cyborg.conf. | |
| 03:59:29 | chenke | I think the current nova's patch is high priority. | |
| 04:00:38 | Sundar | Yumeng and all: Sure. No objections to vGPUs or mediated devices in Cyborg. But it is going to take some effort probably to get that change into Nova | |
| 04:00:51 | shaohe_feng | Yes, the cyborg goal is similar to DPI in k8s, manage different accelerators. | |
| 04:01:48 | shaohe_feng | Nova change for cyborg mdev can later. Let's focus on PCI | |
| 04:01:50 | Sundar | Yumeng: My main comment would be to move 'mdev' type to attach handles from CPIDs. Also, it needs some testing. Do you plan to support 3rd party CI for GPUs? | |
| 04:01:59 | xinranwang | Can vGPU be attached like normal PCI device passthrough, if so, I think there is no conflict. | |
| 04:02:20 | Yumeng | xinranwang: it cannot. | |
| 04:03:22 | Yumeng | xinranwang: nova already support this. we can reuse those code. | |
| 04:03:46 | xinranwang | So VGPU has a different way to be attach, comparing with GPU PF? | |
| 04:04:08 | Yumeng | yes, different way. | |
| 04:04:09 | shaohe_feng | but cyborg can support it. The cloud provider can change it in nova by themselves downstream, if they really want to use cyborg. | |
| 04:04:33 | shaohe_feng | yes, MDEV is uuid in mdev bus. | |
| 04:04:43 | shaohe_feng | PCI is bdf in PCI bus. | |
| 04:04:50 | Yumeng | yes, exactly. MDEV is uuid in mdev bus. | |
| 04:05:14 | shaohe_feng | let cyborg support mdev firstly. | |
| 04:05:43 | shaohe_feng | leave the change in nova to cloud provider. | |
| 04:05:57 | shaohe_feng | they have ability to change it. | |
| 04:06:20 | shaohe_feng | at present. | |
| 04:07:51 | shaohe_feng | they will contribute to nova, if they think it is necessary | |
| 04:08:13 | Sundar | Yumeng: do you plan to support 3rd party CI for GPUs? | |
| 04:08:17 | Yumeng | ok. thanks for the suggestion. shaohe_feng. I think we can talk later about the vGPU. Sundar: pls go back to nova-interaction topic. | |
| 04:08:30 | Yumeng | Sundar: not yet for now. | |
| 04:08:45 | Sundar | ok | |
| 04:09:21 | Sundar | Nova interaction: currently held up on the requirement to support rescheduling. I have a single-node devstack setup, so cannot test rescheduling. | |
| 04:10:55 | Sundar | This does not mean multi-node for CYborg -- the functional tests that I wrote mock the Cyborg APIs, so only need it for the Nova side | |
| 04:11:52 | Sundar | If anybody has any ideas to handle this, please LMK | |
| 04:13:19 | Sundar | That's all I have. Anything else to discuss? | |
| 04:14:11 | Yumeng | ok. not from me. | |
| 04:15:05 | brinzhang | The functional tests by me, I think I should do some search, then to talk, others none of me. | |
| 04:16:09 | Sundar | brinzhang: yes, we need functional tests. You have already started that. What do we need to discuss? | |
| 04:17:22 | brinzhang | Sundar, I think there are need so many work I should consider, wait for enough, I will re take it in ircmeeting | |
| 04:18:10 | brinzhang | Some cleanup patch, can you +A? | |
| 04:18:27 | shaohe_feng | no from me. If you need some help, please ping me on Skype. | |
| 04:18:31 | brinzhang | https://review.opendev.org/#/c/707332/ | |
| 04:18:49 | brinzhang | https://review.opendev.org/#/c/709974/ | |
| 04:19:11 | Sundar | brinzhang: Got it. Sure | |
| 04:19:31 | Sundar | Good. Thanks a lot, everybody. Have a good day! | |
| 04:19:33 | brinzhang | These are not necessary in cyborg or cyborg-specs project, so I want to remove it and make they are clear | |
| 04:19:34 | Sundar | #endmeeting | |
| 04:19:35 | openstack | Meeting ended Thu Feb 27 04:19:34 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 04:19:36 | openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openstack_cyborg.2020-02-27-03.00.html | |
| 04:19:37 | openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openstack_cyborg.2020-02-27-03.00.txt | |
| 04:19:38 | openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openstack_cyborg.2020-02-27-03.00.log.html | |
| 04:19:47 | s_shogo | Only repot: The openstacksdk patch was merged! Thank you,all. https://review.opendev.org/#/c/679914/ | |
| 04:21:26 | Sundar | s_shogo: Congrats! That's good news. It was all your hard work. | |