| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-04-04 | |||
| 14:19:56 | Li_Liu | sure, I will take a look | |
| 14:20:03 | zhipeng | Let's finish the subteam report first | |
| 14:20:32 | zhipeng | Melissa_S chucksong plz refer to https://releases.openstack.org/rocky/schedule.html#r-release | |
| 14:20:41 | zhipeng | for the general milestone sched | |
| 14:21:10 | Sundar | Thanks, Li_Liu and Howard | |
| 14:21:45 | Melissa_S | Yes - thank you. | |
| 14:22:09 | zhipeng | Cyborg's specific sched could be found http://eavesdrop.openstack.org/meetings/openstack_cyborg/2018/openstack_cyborg.2018-03-14-14.07.html | |
| 14:22:28 | zhipeng | okey moving on | |
| 14:22:50 | zhipeng | Yumeng__ could you update the progress on your side ? | |
| 14:23:54 | Yumeng__ | I sent out a update about the high time synchronisation card on the mail list | |
| 14:24:05 | Yumeng__ | Pls check | |
| 14:24:29 | Yumeng__ | And launchpad migration to storyboard will be done soon. | |
| 14:25:00 | zhipeng | did you just send the clock driver proposal ? | |
| 14:25:05 | zhipeng | or maybe I missed it ? | |
| 14:25:50 | Yumeng__ | 1 second | |
| 14:26:36 | zhipeng | BTW in case everyone not follow the storyboard migration, plz refer to | |
| 14:26:39 | zhipeng | #link https://review.openstack.org/#/c/558327/ | |
| 14:26:52 | Yumeng__ | https://docs.openstack.org/infra/manual/zuulv3.html | |
| 14:27:14 | Yumeng__ | Pls go ahead, I will send out the right link | |
| 14:27:28 | zhipeng | #link https://review.openstack.org/558821 | |
| 14:28:02 | zhipeng | #info storyboard migration for cyborg | |
| 14:28:47 | zhipeng | thx Yumeng__ :) doc subteam has been very responsive and active | |
| 14:29:17 | zhipeng | Melissa_S chucksong Dutch signed on as co-lead for the driver sub-team | |
| 14:29:35 | zhipeng | does one of you guys maybe interested in take the role ? | |
| 14:29:52 | zhipeng | help update and coordinate the driver development for Rocky ? | |
| 14:30:31 | Melissa_S | Chuck and I can work on this | |
| 14:30:57 | zhipeng | great :) | |
| 14:31:11 | Melissa_S | I think you could put Chuck's name down - he is on the apps side | |
| 14:31:24 | zhipeng | duly noted :) | |
| 14:31:30 | Melissa_S | we will find the correct internal team for this project | |
| 14:31:35 | zhipeng | #topic critical rocky spec review | |
| 14:32:02 | zhipeng | first up, Sundar's Nova-Cyborg interaction spec | |
| 14:32:22 | chucksong | zhipeng, yes I will take the role | |
| 14:32:25 | zhipeng | Sundar could you help get everyone up to speed ? | |
| 14:33:26 | Sundar | Sure, Howard | |
| 14:33:27 | zhipeng | #info Sundar cyborg-nova spec discussion | |
| 14:33:30 | zhipeng | #link https://etherpad.openstack.org/p/Cyborg-Nova-Multifunction | |
| 14:33:46 | zhipeng | #link https://review.openstack.org/#/c/554717/ | |
| 14:34:01 | Sundar | We had arrived at a way of representing FPGA components in Nova, and worklfows for all use cases based on hat representation, back in the PTG | |
| 14:34:26 | Sundar | The spec that Howard provided above codifies that | |
| 14:35:01 | Sundar | However, there was a race condition for one use case, which has been debated in the community. An analogy was found with a vGPU issue, and a joint solution was proposed | |
| 14:35:27 | Sundar | http://lists.openstack.org/pipermail/openstack-dev/2018-March/128888.html | |
| 14:35:41 | zhipeng | #link http://lists.openstack.org/pipermail/openstack-dev/2018-March/128888.html | |
| 14:36:03 | Sundar | Based on that, I worked out the details, in a way that should cover GPUs etc. but probably focuses more on FPGAs. | |
| 14:36:16 | Sundar | #link https://etherpad.openstack.org/p/Cyborg-Nova-Multifunction | |
| 14:36:38 | Sundar | I think this is a better solution than the PTG flow :) both for Nova and Cyborg | |
| 14:37:34 | Sundar | For Cyborg, this means that we don't need to keep accelerator usage details in Cyborg DB -- just use placement info :) | |
| 14:38:16 | Sundar | Eventually, if/when Nova supports preferred traits, maybe we don't need the weigher -- but we can evaluat that when it becomes real | |
| 14:38:54 | Sundar | For the new folks in Cyborg, this may sound cryptic :) I am open to walking folks through the proposal in a call if needed | |
| 14:39:27 | Sundar | Howard, do you want me to explain the crux of the proposal here? | |
| 14:39:28 | Li_Liu | Sundar, regardless, we still need to track individual assignment for each device in Cyborg, right? | |
| 14:40:04 | zhipeng | Sundar it would be great to have a brief walk through and the central take aways :) | |
| 14:40:07 | Sundar | Li_Liu: Yes, we will be handling acceleratr -PF/VF matching | |
| 14:40:46 | Sundar | That is not involve din the scheduling, because Placement counts accelerator resources | |
| 14:41:16 | Sundar | As instances are spawned and terminated, Cyborg agent can track PF/VF usage | |
| 14:41:50 | Sundar | Howard, sure. | |
| 14:41:56 | Sundar | The brief summary is this: | |
| 14:42:13 | Melissa_S | Sundar - I am open to scheduling a call and walkthrough of your proposal. | |
| 14:42:32 | Sundar | We want to expose FPGAs in 2 ways: FPGA as a Service (FPGA aaS), and Accelerated Function as a Service (AFaaS) | |
| 14:43:20 | Sundar | For FPGAaaS, user/flavor asks for a device type (or region type) and optionally a bitstream ID that needs to be applied by Cyborg before the instance comes up. | |
| 14:43:44 | Sundar | This is similar to Amazon flow. | |
| 14:44:19 | Sundar | For AFaaS, the user/flavor asks for a function/algorithm e.g. 'ipsec' + some indication of what device family the VM has the drivers for | |
| 14:44:42 | Sundar | To cover both cases, we say that an accelerator can be a device/region or a function | |
| 14:45:09 | Sundar | We represent a generic accelerator with the custom resource class (RC) CUSTOM_ACCELERATOR | |
| 14:45:34 | Sundar | We also represent FPGAs and their inner regions as nested Resource Providers (RPs) | |
| 14:45:57 | Sundar | So, a region RP can provide N instances of a CUSTOM_ACCELERATOR class. Most commonly, N=1 | |
| 14:46:48 | Sundar | Also, each region RP has traits: region type (e.g. CUSTOM_FPGA_<vendor>_REGION_...) | |
| 14:47:08 | Sundar | possibly function type (e.g. CUSTOM_FPGA_<ipsec_uuid>) | |
| 14:47:37 | Sundar | and device family (e.g. CUSTOM_FPGA_XILINX_<product/part>...) | |
| 14:47:52 | Li_Liu | Sundar, for AFaaS, on top of asking for a function/algorithm, I think users can also specify the minimum kpi/capability for the requested resources | |
| 14:48:05 | Li_Liu | you keep going | |
| 14:48:15 | Li_Liu | I just throw some of my thoughts here | |
| 14:48:22 | Sundar | Li_Liu, NP, I'll get back to that | |
| 14:48:51 | Sundar | With this background, here's how a flavor can ask for FPGA aaS: | |
| 14:49:24 | Sundar | resource:CUSTOM_ACCELERATOR=1; trait:CUSTOM_FPGA_<vendor>_REGION_<uuid>=required | |
| 14:49:48 | Sundar | optionally, one more extra spec: bitstream:3A56D4=required | |
| 14:50:29 | Sundar | This gets Placement to choose all matching devices based on trait. Once a node is selected, the Cyborg agent in the node notes the extra spec and applies the bitstream | |
| 14:50:44 | Sundar | Before I go to AFaaS, does this make sense for FPGA aaS/ | |
| 14:50:57 | zhipeng | it does for me, this is awesome Sundar | |
| 14:51:19 | Li_Liu | yup, it looks great | |
| 14:51:34 | Sundar | Cool :) Now for AFaaS flavor | |
| 14:51:43 | chucksong | It sounds good Sundar. I need to know more about this project though. I just joined this project and still have a lot to learn | |
| 14:52:22 | Duth | Yes agreed. Sundar, can you share some more information on this proposal for FPGA aaS/ | |
| 14:52:42 | zhipeng | Duth let's schedule another dedicated call for it | |
| 14:52:45 | zhipeng | :) | |
| 14:52:57 | Sundar | resource:CUSTOM_ACCELERATOR=1; trait:CUSTOM_FPGA_INTEL_<ipsec-uuid>=required; trait:CUSOTM_FPGA_INTEL_<product>=required | |
| 14:52:59 | zhipeng | video conf would be better for a deep dive walk through | |
| 14:53:18 | Melissa_S | Agreed. | |
| 14:53:18 | Sundar | Yes, we can do more in a video call | |
| 14:53:39 | zhipeng | Sundar plz carry on the AFaaS scenario | |
| 14:54:10 | Sundar | Some folks don't like the UUIDs. I think UUIDs make it very concrete and we can make it more user-friendly later. Also, for AFaaS, we can avoid region UUIDs and use just product name | |
| 14:54:37 | zhipeng | string name could do the trick ? | |
| 14:54:37 | Sundar | OK, back to AFaaS :) | |
| 14:54:50 | zhipeng | sorry go ahead | |
| 14:56:50 | Sundar | Zhipeng: We can discuss strings in more deatail, may be in an email? | |
| 14:57:19 | zhipeng | Sundar absolutely | |
| 14:57:38 | zhipeng | plz carry on, sorry for the interruption :) | |
| 14:57:39 | Sundar | OK. AfFaaS flavor: resource:CUSTOM_ACCELERATOR=1; trait:CUSTOM_FPGA_INTEL_<ipsec-uuid>=required; trait:CUSOTM_FPGA_INTEL_<product>=required | |
| 14:58:37 | Sundar | This picks all devices that have the required function. Makes sense, except that if no free instance of that function is available, the request will just fail, rather than have Cyborg pick an available region and reprogram it | |
| 14:59:18 | Sundar | That is ok if that's what the operator wants: he may want to prevent reprogramming for whatever reason. | |