| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-04-04 | |||
| 14:50:29 | Sundar | This gets Placement to choose all matching devices based on trait. Once a node is selected, the Cyborg agent in the node notes the extra spec and applies the bitstream | |
| 14:50:44 | Sundar | Before I go to AFaaS, does this make sense for FPGA aaS/ | |
| 14:50:57 | zhipeng | it does for me, this is awesome Sundar | |
| 14:51:19 | Li_Liu | yup, it looks great | |
| 14:51:34 | Sundar | Cool :) Now for AFaaS flavor | |
| 14:51:43 | chucksong | It sounds good Sundar. I need to know more about this project though. I just joined this project and still have a lot to learn | |
| 14:52:22 | Duth | Yes agreed. Sundar, can you share some more information on this proposal for FPGA aaS/ | |
| 14:52:42 | zhipeng | Duth let's schedule another dedicated call for it | |
| 14:52:45 | zhipeng | :) | |
| 14:52:57 | Sundar | resource:CUSTOM_ACCELERATOR=1; trait:CUSTOM_FPGA_INTEL_<ipsec-uuid>=required; trait:CUSOTM_FPGA_INTEL_<product>=required | |
| 14:52:59 | zhipeng | video conf would be better for a deep dive walk through | |
| 14:53:18 | Melissa_S | Agreed. | |
| 14:53:18 | Sundar | Yes, we can do more in a video call | |
| 14:53:39 | zhipeng | Sundar plz carry on the AFaaS scenario | |
| 14:54:10 | Sundar | Some folks don't like the UUIDs. I think UUIDs make it very concrete and we can make it more user-friendly later. Also, for AFaaS, we can avoid region UUIDs and use just product name | |
| 14:54:37 | zhipeng | string name could do the trick ? | |
| 14:54:37 | Sundar | OK, back to AFaaS :) | |
| 14:54:50 | zhipeng | sorry go ahead | |
| 14:56:50 | Sundar | Zhipeng: We can discuss strings in more deatail, may be in an email? | |
| 14:57:19 | zhipeng | Sundar absolutely | |
| 14:57:38 | zhipeng | plz carry on, sorry for the interruption :) | |
| 14:57:39 | Sundar | OK. AfFaaS flavor: resource:CUSTOM_ACCELERATOR=1; trait:CUSTOM_FPGA_INTEL_<ipsec-uuid>=required; trait:CUSOTM_FPGA_INTEL_<product>=required | |
| 14:58:37 | Sundar | This picks all devices that have the required function. Makes sense, except that if no free instance of that function is available, the request will just fail, rather than have Cyborg pick an available region and reprogram it | |
| 14:59:18 | Sundar | That is ok if that's what the operator wants: he may want to prevent reprogramming for whatever reason. | |
| 14:59:37 | Sundar | But, in the general case, we want the ability to reprogram if needed. | |
| 15:00:04 | Sundar | So, what we want is: trait:CUSTOM_FPGA_INTEL_<ipsec-uuid>=preferred (not required) | |
| 15:00:34 | Sundar | But Nova does not support preferred traits today. I am told that it is not even close | |
| 15:01:01 | Sundar | So, the next best thing: resource:CUSTOM_ACCELERATOR=1; trait:CUSOTM_FPGA_INTEL_<product>=required | |
| 15:01:19 | Sundar | another extra spec: function:CUSTOM_FPGA_INTEL_<ipsec-uuid>=required | |
| 15:01:45 | Sundar | Now, Placement chooses all devices that match the product, whether or not they have the function | |
| 15:02:21 | Sundar | A Cyborg weigher can check the allocation candidates to see which ones have the function -- based on the function trait -- and rank them higher | |
| 15:03:13 | Sundar | So, Nova is likely to pick a device that has the function. if not, Cyborg agent in the compute node will note the requested function is not present in the selected device/region RP | |
| 15:03:35 | Sundar | It will contact Glance to get a matching bitstream and program it, before the instance comes up | |
| 15:03:46 | zhipeng | where would the weigher be implemented ? | |
| 15:04:06 | Sundar | This is the core of the proposal, and covers situations where each bitstream implements only one function | |
| 15:04:48 | Sundar | Zhipeng: the weigher would initially be a Cyborg weigher in my understanding -- but it is not Cyborg-specific. It could possibly become Nova in-tree hopefully | |
| 15:05:25 | Sundar | The weighe ris looking at a list of RPS and choosing those with a specific trait. It is generic | |
| 15:05:37 | zhipeng | i mean the weigher will be part of the agent ? | |
| 15:06:10 | Sundar | It is a weigher in the Nova controller -- like all other filters/weighers in Nova framework | |
| 15:06:21 | Li_Liu | The weigher has to be in Cyborg Controller I think | |
| 15:07:06 | shaohe_feng_ | yes, it should. And seems nova agree on the weigher | |
| 15:07:18 | Sundar | Li_Liu: OK, what I meant is, it runs in the controlle ralong with Nova/Cyborg. The operator must update nova.conf to use this weigher | |
| 15:07:36 | Li_Liu | Right :) | |
| 15:07:47 | zhipeng | Okey | |
| 15:08:15 | shaohe_feng_ | yes. just a config option in nova | |
| 15:08:45 | zhipeng | Sundar I think it is definitely fine to update the spec patch based upon the current discussion conclusion | |
| 15:09:04 | zhipeng | and let's schedule another video conf for a detailed discussion with Xilinx team | |
| 15:09:08 | Sundar | The part of this proposal which needs further review from Nova is when one bitstream has multiple functions, say crypto and compression. I think that is for the future and may not be needed in rocky. Does that sound agreeable? | |
| 15:09:15 | zhipeng | to see if there is any further improvements | |
| 15:09:37 | shaohe_feng_ | but weigher may be not very high priority. It is help to speed up the creation of a VM. but no helpful for the VM performance after VM start | |
| 15:09:39 | Sundar | Thanks, Howard! | |
| 15:09:40 | zhipeng | Sundar yes that would be maybe next release :) | |
| 15:10:28 | edleafe | Just an FYI - it's not likely that nested RPs will be complete in Rocky | |
| 15:10:39 | zhipeng | shalhe_feng_ agree, let's implement the custom rc and traits first | |
| 15:10:54 | zhipeng | edleafe it is possible that we try with the nrp first right ? | |
| 15:11:32 | edleafe | zhipeng: sure, but it looks like the earliest nrp will be available will be in Stein | |
| 15:11:46 | Sundar | edleafe: There is a release notes in Queens for nRPs, right? https://github.com/openstack/nova/blob/adc4d4a29d108c87f884c779af5696e4941b9549/releasenotes/notes/placement-rest-api-nested-resource-providers-552a923a96d7adca.yaml | |
| 15:12:42 | edleafe | Sundar: that is the very beginnings of the structural changes needed for nrp | |
| 15:13:10 | shaohe_feng_ | edleafe: so cyborg just can support a fpga resource class in node provider in this release? | |
| 15:13:19 | edleafe | the full model we need is still far away | |
| 15:13:44 | Sundar | edleafe: The backup optoion would be to apply the RCs and traits to the compute node RP. But, when there are multiple devices in the same node, that can result in issues. | |
| 15:13:53 | edleafe | shaohe_feng_: I'm not sure how that would work if you have multiple devices per node | |
| 15:14:25 | Sundar | edleafe: We crossed. :) | |
| 15:14:29 | edleafe | I'm on a call right now - I just wanted to set your expectations | |
| 15:14:38 | zhipeng | edleafe thx :) | |
| 15:14:39 | shaohe_feng_ | edleafe: IMHO, we can support multiple devices later. | |
| 15:14:56 | zhipeng | yes let's be flexible | |
| 15:15:27 | zhipeng | #action Sundar update the spec according to the ml discussion conclusion | |
| 15:15:37 | Sundar | We can support multiple devices with some restrictions, which may satisfy immediate needs and still give freedom to operaors | |
| 15:15:52 | zhipeng | I would especially thanks Sundar for his initiative on the mailing list | |
| 15:16:06 | shaohe_feng_ | Sundar: any code plan for the spec? | |
| 15:16:08 | zhipeng | and also the spec discussion with Nova team | |
| 15:16:14 | Li_Liu | Or can we put a simple version of multiple device support in Cyborg for now? | |
| 15:16:21 | Sundar | shaohe, yes. | |
| 15:16:25 | zhipeng | shaohe-feng_ your PoC code could be used right ? | |
| 15:16:27 | Sundar | Thanks, Howard | |
| 15:16:54 | shaohe_feng_ | zhipeng: yes. I think so. | |
| 15:16:59 | Sundar | The POC code does not publish RCs and traits | |
| 15:17:04 | Sundar | But we can build on that | |
| 15:17:17 | zhipeng | yes that's what i meant | |
| 15:17:20 | shaohe_feng_ | Sundar: it publish | |
| 15:17:40 | Sundar | Also, the notion of using PFs and VFs as resources is something the Nova/PTG folks didn;t want ;) | |
| 15:18:04 | Sundar | Shaohe: sorry, to clarify, it publishes PCI functions as RCs right? | |
| 15:18:12 | zhipeng | let's take the details offline :) | |
| 15:18:20 | Sundar | ok :) | |
| 15:18:22 | shaohe_feng_ | Sundar: the poc is similar to nova teams conclusion. | |
| 15:18:27 | zhipeng | next up, Li Liu's metadata spec | |
| 15:18:38 | zhipeng | #info metadata standardization spec | |
| 15:18:50 | zhipeng | #link https://review.openstack.org/558265 | |
| 15:19:03 | shaohe_feng_ | and we know nrp is not ready, so simply it. | |
| 15:19:04 | zhipeng | folks plz review it | |
| 15:19:13 | zhipeng | anything you want to add, Li_Liu ? | |
| 15:19:33 | chucksong | zhipeng, sorry for interrupt, what's the expected freeze date for the Xilinx driver spec? april 19 or Jun.4? | |
| 15:19:51 | zhipeng | chucksong Jun 4 | |
| 15:19:55 | Li_Liu | zhipeng, I made some modifications based on shaohe's comments couple days ago. Waiting for more suggestions | |
| 15:20:08 | zhipeng | Li_Liu okey :) | |
| 15:20:09 | chucksong | good, thanks! | |
| 15:20:21 | zhipeng | okey moving on | |
| 15:20:27 | Li_Liu | My next spec for programmability is on the way. within the week I think | |
| 15:20:44 | zhipeng | Li_Liu you are the rock star man | |