| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-06-27 | |||
| 14:19:25 | Li_Liu | right | |
| 14:20:08 | Sundar | OK, next steps are to add a sequence diagram. Also, I would propose a plugin for x86 + FPGAs, and a separate one for x86 + GPUs. The main reason to stick to x86 is that that's all I know about :) | |
| 14:20:42 | Sundar | We can probably generalize to ARM if SR-IOV works the same way there. Power is an area where we need Eric's input | |
| 14:20:59 | Li_Liu | how much difference if it was arm or even power? | |
| 14:21:00 | efried | I don't think you should be proposing specific plugins via this spec. | |
| 14:21:06 | efried | just the framework. | |
| 14:21:24 | efried | The plugins themselves will ultimately be the responsibility of the vendors. | |
| 14:21:29 | efried | I would think. | |
| 14:21:58 | Sundar | For x86+FPGAs, we could have a community-supported plugin which calls into Cyborg/agent/driver for vendor-specific stuff | |
| 14:22:15 | Li_Liu | shall we provide a reference from community tho? | |
| 14:22:28 | Sundar | Li_Liu: yes ^^ | |
| 14:23:06 | Sundar | Li_Liu: "how much difference if it was arm or even power?" Power seems to be quite different IIUC. Eric can expand on that | |
| 14:23:07 | shaohe_feng | entry points for driver. | |
| 14:24:32 | efried | Yeah, e.g. if the plugin has a "plug" operation, on libvirt this entails, what, editing the domain XML and doing stuff to special files under /dev? | |
| 14:25:01 | efried | In POWER, it entails making a REST call to the NovaLink API asking for an I/O slot to be attached to a logical partition (VM) and the platform does the magic. | |
| 14:25:07 | efried | Totally, totally different model. | |
| 14:25:25 | efried | and definitely not something the cyborg team should try to become domain experts on. | |
| 14:26:31 | Sundar | efried: Agreed. The plugin model should be able to handle such differences. The return value of the plug() itself is left open for that reason. | |
| 14:26:55 | Li_Liu | sounds good | |
| 14:27:01 | efried | I would think the return value would be an Acc* object | |
| 14:27:13 | Sundar | That is why it may make sense to have a x86/ARM/libvirt section that specifies what plug(), does | |
| 14:27:33 | efried | If it needs to contain platform-specific data, that goes in the *Profile bit, or something. | |
| 14:28:01 | Sundar | The return value can be a VAN object with enough data that is hypervisor-specific | |
| 14:28:20 | efried | Sundar: sorry, yeah, I guess we're calling it VAN now. | |
| 14:28:41 | Sundar | The port profiles in Neutron may not be applicable here IMHO | |
| 14:28:52 | efried | But again, I don't think your spec should go into any detail as to what each platform's plugin actually does. You could mention it for the sake of example/understanding, but it should not be the role of this spec to lay out the details. | |
| 14:29:32 | Sundar | OK. If we agree that we will do a community-supported plugin for x86 + FPGAs, I can write a separate spec for that. Is that fine? | |
| 14:30:41 | efried | yes, that sounds like the right plan. | |
| 14:31:14 | efried | Sundar: update on the seqdiag: right now it looks like the cyborg repo isn't yet using the "new" process for doc builds. | |
| 14:31:33 | efried | So you'll want to put the requirement into test-requirements.txt instead of creating a new doc/requirements.txt | |
| 14:31:35 | Sundar | Li_Liu, shaohe, all: do we all agree on this plan? | |
| 14:31:39 | efried | or you can switch over to the new build process | |
| 14:32:02 | Sundar | Ah, I put it in cyborg/requirements.txt | |
| 14:32:35 | Sundar | That seems closest? | |
| 14:32:53 | Li_Liu | Sundar, i am fine | |
| 14:33:02 | efried | Right now your docs env is calling out test-requirements.txt | |
| 14:33:09 | efried | I don't actually see anything calling out requirements.txt, which is weird. | |
| 14:35:07 | efried | Sundar: Anyway, I'm noodling around with it locally, will let you know if I get something working. | |
| 14:35:23 | Li_Liu | let's moving on | |
| 14:35:45 | Sundar | efried: Sorry, where do you see the reference to test-requirements.txt? | |
| 14:36:48 | Sundar | I mean , I see the file but what in doc is referring to it? | |
| 14:37:02 | Li_Liu | Cyborg had a meetup in Beijing LC3 yesterday, shaohe_feng do you have anything to share? | |
| 14:38:50 | shaohe_feng | zhipengh[m], introduce some scenario about cyborg | |
| 14:39:42 | Li_Liu | #topic LC3 meetup summary | |
| 14:43:52 | Li_Liu | ok, I will gather more information from the folks who attend the meetup and share with rest of the team | |
| 14:44:09 | Li_Liu | #topc AoB | |
| 14:44:14 | Li_Liu | #topic AoB | |
| 14:44:32 | shaohe_feng | Do you know iflytek? Sundar, Li_Liu | |
| 14:44:59 | Li_Liu | I know them | |
| 14:45:18 | shaohe_feng | #link http://www.iflytek.com/en/index.html | |
| 14:45:18 | Sundar | This one? https://en.wikipedia.org/wiki/IFlytek | |
| 14:46:01 | shaohe_feng | It is a famous AI company in China. | |
| 14:46:08 | shaohe_feng | Sundar, yes. | |
| 14:47:04 | shaohe_feng | AI will be a cyborg scenario. | |
| 14:47:12 | shaohe_feng | they need accelerator | |
| 14:47:46 | shaohe_feng | then we discuss the current status of cyborg development. | |
| 14:47:47 | Sundar | GPUs? | |
| 14:48:06 | shaohe_feng | Sundar, Gpus and FPGA. | |
| 14:48:08 | wangzhh | FPGAs, also. :) | |
| 14:48:13 | Li_Liu | are they planing to put people on Cyborg? | |
| 14:48:41 | shaohe_feng | Not sure. | |
| 14:49:49 | shaohe_feng | Sundar, they develop logic on FPGA. | |
| 14:50:17 | shaohe_feng | we have a gap on cyborg API design. | |
| 14:50:31 | Sundar | Interesting. So, they have their own bitstreams. How are they using it -- like FPGA aaS? | |
| 14:52:13 | shaohe_feng | Not sure how do they use it. | |
| 14:52:34 | shaohe_feng | Sundar, will check with them about it by mail. and CC you. | |
| 14:52:42 | shaohe_feng | for we need to interactive nova. | |
| 14:53:13 | Sundar | shaohe: Ok | |
| 14:53:42 | shaohe_feng | wangzhh, introduce the whole flow for the interaction. | |
| 14:53:55 | shaohe_feng | they have do a demo for it during summit. | |
| 14:54:18 | wangzhh | OK. We have a discussion offline this afternoon. | |
| 14:54:34 | wangzhh | Let me introduce about it. | |
| 14:54:49 | shaohe_feng | wangzhh, can you give a more details on it? | |
| 14:55:31 | wangzhh | It is about the flow of creating, deleting and rebooting instance with accelerators. | |
| 14:55:41 | wangzhh | Create: | |
| 14:55:54 | wangzhh | 1. nova-api (Create an instance with acc) | |
| 14:56:00 | wangzhh | 2. nova-conductor => nova-scheduler => placement-api (Get a node list and claim the accelerator unit) | |
| 14:56:05 | wangzhh | 3. nova-conductor => nova-compute | |
| 14:56:11 | wangzhh | Extra work we should do in nova: | |
| 14:56:19 | wangzhh | a) nova-compute => cyborgclient => cyborg-api (Update acc, instance_uuid, assignable,etc. Program here if it is a request of fpga) | |
| 14:56:27 | wangzhh | b) nova-compute pass parameters to os-acc(accelerator_address, acc_type, etc.) and get xml segment | |
| 14:57:36 | Sundar | Why do they do "nova-compute => cyborgclient => cyborg-api" for programming? We can do it entirely in compute node | |
| 14:58:14 | Sundar | We need to update the db, and that requires an RPC to the conductor | |
| 14:58:34 | Sundar | But, otherwise, it can all be in os-acc/Cyborg-agent/driver | |
| 14:58:36 | wangzhh | It is for 2 reasons. | |
| 14:58:37 | efried | I'll just mention that there was a nova core (it might have been Dan) who said nova-compute should never have to call the cyborg API; everything should be done via os-acc. | |
| 14:58:41 | efried | I could be misquoting too :( | |
| 15:00:09 | Sundar | wangzhh: Did they modify Cyborg? | |
| 15:00:41 | wangzhh | 1. In Openstack, It is a common design to interact by api. 2. We have something like update quota in api. | |
| 15:02:01 | Sundar | wangzhh: If Nova compute has to call the REST APIs of Neutron, Cinder, Cyborg, ... it can get out of hand. The os-acc/os-vif/os-brick is supposed to solve that problem | |
| 15:03:03 | wangzhh | efried, if so, could we imply it by os-acc =》 cyborclient? | |
| 15:03:22 | efried | wangzhh: Yes, I would think so. | |
| 15:03:42 | Sundar | efried: How about Power? Would you have os-acc call Cyborg for that too? | |
| 15:04:09 | efried | Sundar: I would think any calls from os-acc to cyborg API would happen *outside* of the plugin, in the common os-acc code. | |
| 15:04:33 | wangzhh | Sundar, IMHO os-vif/os-brick didn't call neutron or cinder. | |
| 15:05:20 | efried | I probably need to punt on this issue, just wanted to bring it up so y'all didn't get miles down the road and end up surprised if someone started waving a red flag. | |
| 15:05:51 | Sundar | In the currently proposed flow, os-acc --> plugin, and the plugin can do whatever it wants. For FPGAs, I think it would call Cyborg agent, which in turn can call the Cyborg conductor etc. | |
| 15:06:26 | Sundar | For Power plugin, there could be a common os-acc function which invokes cyborg client to update the API | |
| 15:06:38 | wangzhh | I think API is the entry of cyborg. | |
| 15:07:12 | wangzhh | If we call agent directly. Some process would be complex. | |