| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2020-06-11 | |||
| 03:09:50 | wangzhh | I didn't follow for a long time, just listen at first.... | |
| 03:10:04 | Yumeng | yes, agree. I may miss some in the PTG. But agree. | |
| 03:10:21 | chenke | Do you means the first case is only one region ,right? | |
| 03:11:07 | Sundar | The concept of region is specific to FPGAs. A smart NIC may not have FPGAs. May be it is a NIC ASIC + many ARM cores. | |
| 03:11:30 | s_shogo | As a first step, that is good. agree. | |
| 03:11:52 | xinranwang | It may have fpga as well. I think we need consider that. I think subprovider can support this. | |
| 03:11:54 | Sundar | Great. For the second case, we could have a hierarchy with separate RPs for the accelerator and the NICs, and a top-level resource-less RP which aggregates all the children RPs and combines their traits | |
| 03:12:33 | Sundar | xinranwang: Of course. My point was, the concept of region can be used if it is a FPGA, not otherwise. | |
| 03:12:55 | chenke | Sundar ok. got it. | |
| 03:13:35 | Sundar | For example: N3000 may be modeled as two RPs - one for the FPGA and one for the 2 Fortville NICs. Perhaps each Fortville NIC cna be a separate RP. That is up to us. | |
| 03:13:54 | xinranwang | I agree. | |
| 03:14:49 | Sundar | Correspondingly, Cyborg will create a separate deployable for each component, and a top-level deployable that contains all component deployables. | |
| 03:15:38 | Sundar | The resources and traits of the children RPs are exposed in the top-level RP; similarly, the accelerators and attributes of children deployables are in the top-level deployable. | |
| 03:16:03 | xinranwang | It corresponds to the device topology. Cyborg can report like this. If we need interact with other project like neutron, we should consider more. Because neutron report to placement if bandwidth feature is enabled. | |
| 03:17:00 | Sundar | xinranwang: I am hoping that Cyborg can create RPs for all components, not leave it to neutron even for bandwidth provider. Otherwise, it gets a bit complicated. | |
| 03:17:20 | Sundar | Neutron can, of course, use the RP that Cyborg created | |
| 03:17:49 | xinranwang | The physnet is neutron who's in charge of, not sure cyborg should take it over. | |
| 03:18:10 | xinranwang | it a network concept | |
| 03:18:16 | xinranwang | s/it/it's | |
| 03:18:19 | Sundar | Yes. The physnet is best left to the admin or the OpenStack installer | |
| 03:18:47 | Sundar | Cyborg shouldn't get in the way. | |
| 03:19:23 | Sundar | However, we could model the physnet as a trait. Cyborg creates the NIC's RP but doesn't want to manage the physnet trait on that RP. | |
| 03:19:37 | xinranwang | what is neutron's bandwidth feature enabled, how will cyborg know that neutron report to placement as well. | |
| 03:19:44 | xinranwang | s/what is/what if | |
| 03:19:56 | Sundar | So, we can levae it to the admin or installer however they do it. That is how PCI whitelist handles physnet anyway. | |
| 03:20:47 | xinranwang | I am not sure this will be accepted by the community. I am still thinking about it. | |
| 03:21:03 | Sundar | xinranwang: Good point, I have thought about it. Hoping we can reach agreement with Neutron that they need not create RPs in this case. | |
| 03:21:51 | Sundar | What won;t be accepted:: phsynet as trait, or Cyborg creating the RPs for NICs? | |
| 03:22:32 | xinranwang | Anyway, I will propose a spec in nova community and everyone can discuss there. But I am still thinking about which solution I should choose to propose firstly, and others will be an alternative. | |
| 03:23:19 | Sundar | We should have some internal agreement hopefully before we approach others | |
| 03:24:39 | xinranwang | Yes, of course | |
| 03:24:42 | Sundar | Anyways, what do others think of Cyborg creating RPs for the NIC side too? It will do that for the first type where the accelerator and NIC are combined. To keep it uniform, we should do that for the second case too | |
| 03:25:16 | Sundar | Otherwise, we'll have different solutions for different types of smart NICs, depending on whether they have a single component or multiple components | |
| 03:26:36 | Sundar | Yumeng, chenke, s_shogo, all: ^ | |
| 03:27:42 | Yumeng | so if Admin use Placement CLI to set traits, then admin should know “physicalnet” and RP RC of this smartNIC. so does that mean admin needs to GET physicalnet first, then GET RP,RC, then report? | |
| 03:27:43 | Sundar | Anyway, I'll throw another idea in. Ideally, the admin should be able to formulate the device profile in the same way, independent of whether it is a single-component or multi-component device. | |
| 03:28:36 | xinranwang | s_shogo: | |
| 03:28:42 | Sundar | Yumeng: the admin needs to get the RP for the NIC and set the trait there. he could do so via the installer | |
| 03:28:48 | xinranwang | Shogo, are you around? | |
| 03:29:35 | s_shogo | xinranwang: yes, I'm considering about that, with thinking about the operation of N3000.. | |
| 03:29:59 | Sundar | Good. Thanks, s_shogo | |
| 03:30:07 | Sundar | To repeat: Ideally, the admin should be able to formulate the device profile in the same way, independent of whether it is a single-component or multi-component device. | |
| 03:30:23 | Sundar | That common device profile would look like this: | |
| 03:30:33 | chenke | Sundar: I am not familar with this. | |
| 03:30:58 | xinranwang | s_shogo: lol, np. Just for other things. Haibin can not access to IRC, and he met some problem with this https://review.opendev.org/#/c/698190/ , could you connect him, maybe by email? | |
| 03:31:45 | Sundar | chenke: ok, np | |
| 03:32:01 | Sundar | { "name": "my-smartnic-dp", | |
| 03:32:03 | s_shogo | Yes, I agree that from operator 's point of view > same way to treat device profile | |
| 03:32:18 | Sundar | { "name": "my-smartnic-dp", | |
| 03:32:26 | Sundar | "groups": [{ | |
| 03:32:35 | Sundar | "resources:FPGA": "1", | |
| 03:32:46 | Sundar | "resources:CUSTOM_NIC_X": "1", | |
| 03:33:05 | Sundar | "trait:CUSTOM_FPGA_REGION_ID_FOO": "required", | |
| 03:33:15 | Sundar | "trait:CUSTOM_NIC_TRAIT_BAR": "required", | |
| 03:33:22 | Sundar | "accel:bitstream_id": "3AFE" | |
| 03:33:29 | s_shogo | xinranwang: OK, I took conversation with haibin and shaohe yesterday, that seems to be solved. Of course, If another one , I can help that, too. | |
| 03:33:31 | Sundar | }] | |
| 03:33:32 | Sundar | } | |
| 03:33:51 | Sundar | IOW, the resource, traits and Cyborg properties for both the accelerator and NIC would be presented as a single resource group, which would ensure that a single RP would have to satisfy that. That single RP could be the top-level RP of a hierarchy. | |
| 03:34:13 | shaohe_feng | s_shogo: yes, another one, need your help | |
| 03:34:35 | xinranwang | s_shogo: thanks, please contace him when you got time. | |
| 03:34:41 | Sundar | Basically, unless it is a single request group, it is not guaranteed that the resources will come form the same RP | |
| 03:35:03 | Sundar | Do you have any comments or questions? | |
| 03:35:11 | s_shogo | shaohe_feng, OK,I got it! Could you contact me via e-mail like yesterday? | |
| 03:35:25 | shaohe_feng | s_shogo: yes. | |
| 03:35:46 | Sundar | During ARQ binding, Cyborg would still get a single RP as today. In the case of a multi-component device, Cyborg would translate that to the top-level Deployable object, and figure out what constituent components are present. | |
| 03:35:50 | xinranwang | we can merge the triat to a single request group, from cyborg, or from neutron. | |
| 03:36:48 | xinranwang | I am not against you Sundar I understand your proposal, it is one of the solution we presents in PTG. | |
| 03:36:55 | Sundar | Neutron cannot add traits to a RP owned by Cyborg, as per existing agreement among developers. | |
| 03:38:24 | Sundar | I don't think we got into the aspects of how the device profile should look, during the PTG. It is impotant to nail that down, so we get a uniform way of handling all smart NICs, whether they have 1 component or many | |
| 03:38:53 | xinranwang | If neutron don't want do this. Cyborg can update traits created by neutron, in that case, cyborg should know whether neutron create RP or not. We should think about it. | |
| 03:39:47 | Yumeng | Sundar,xinranwang: I just got a question. if admin knows the "physicalnet", why can't the admin create a device_profile with "physicalnet" directly? | |
| 03:39:50 | Sundar | If one OpenStack service starts updating traits on RPs created by another service, that can cause confusion. You are welcome to discuss it with other developers. | |
| 03:40:06 | Sundar | We have already had many such discussions during Nova-Cyborg discussion. | |
| 03:40:13 | xinranwang | How device_profile looks like is not related to how we report the RP. It's 2 questions. | |
| 03:40:36 | xinranwang | Yumeng: yes, admin can do that. | |
| 03:40:39 | Yumeng | so that either nova or neutron does not have to merge "physicalnet" trait to request group | |
| 03:40:53 | swp20 | Sundar, 'During ARQ binding, Cyborg would still get a single RP as today.' the RP is for FPGA or NIC_X? | |
| 03:41:13 | Sundar | My point is, the device profile must have a single RG for both sides. If you create 2 separate RGs for the multi-component case, it won't ensure co-location. | |
| 03:41:39 | Sundar | swp20: That would be the top-level RP, which contains both the FPGA and NIC RPs as children. | |
| 03:41:40 | xinranwang | I think we have mix 2 questions... one is who report to placement, one is how nova get this traits before scheduling. | |
| 03:41:49 | Yumeng | xinranwang: ok. so that's also one of the solution. | |
| 03:42:42 | Sundar | Yumeng: admin can create device profiles with physnet as a trait. The question is, how is that trait set in the RP, before it is referenced in the devic eprofile. | |
| 03:43:17 | Sundar | xinranwang: No, I have addressed both questions separately above. | |
| 03:45:06 | swp20 | Sundar: got it,Thanks. | |
| 03:45:19 | xinranwang | Yumeng: yes, sure. That's one of the solution | |
| 03:46:00 | Sundar | flavor to do the scheduling. | |
| 03:46:00 | Sundar | If it is not clear, let me state it again: I think Cyborg should create all the RPs needed for a smart NIC. It populates most traits. Some traits, like physnet, may need to come from the admin. The dmin creates device profiles. The user/tenant creates Neutron ports with the device profile name. Nova uses the device profiles in neutorn ports + | |
| 03:46:50 | xinranwang | Yes, I agree. That's one of the solution | |
| 03:47:17 | Sundar | What alternative solution is under consideration? | |
| 03:47:35 | xinranwang | ok, let me state | |
| 03:48:15 | xinranwang | 1. cyborg create rp, neutron use this rp and update phynet traits. | |
| 03:48:16 | swp20 | Sundar: ' Some traits, like physnet', should we report these traits to Placement? | |
| 03:48:43 | xinranwang | 2. neutron create rp, cyborg use this rp to update acc related rc and traits. | |
| 03:49:51 | Sundar | First, you are assuming it is ok for one service to create an RP and another to update it by adding traits. Secondly, who will create the top-level RP for multi-component NICs? | |
| 03:51:28 | xinranwang | there is also solution by using provider-config.yaml | |
| 03:51:31 | Sundar | swp20: SOme traits cna be discovered from the device's PCI ID, like the type of NIC. Those can be added as traits by Cyborg. Others, like physnet or external network connecivity, are know only to the admin or the OpenStack installer. They are best left out of Cyborg. | |
| 03:51:45 | Sundar | The provider-config.yaml doe snot create RPs today. | |
| 03:52:07 | Sundar | It can be enhanced n the future to do that, but that will be another long discussion and development | |
| 03:52:16 | xinranwang | cyborg create rp and other update traits. | |