| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2017-06-07 | |||
| 15:08:20 | zhipeng_ | from the email link Chris provided, there is an etherpad documenting the "Plan B" | |
| 15:09:02 | zhipeng_ | which I liked very much, is working on to extend the current nested resource provider definition, to a more relaxed, multiple resource providers one | |
| 15:09:02 | jkilpatr | ok so the issue is that if we have a 'gpu' resource provider it's dependent on computes in a way that resource providers aren't supposed to be. | |
| 15:09:11 | zhipeng_ | yes exactly | |
| 15:09:42 | zhipeng_ | the scheduling decision would still largely depends on the regular compute features, since we are just part of the traits | |
| 15:09:59 | crushil | interesting | |
| 15:10:12 | zhipeng_ | so back to the "Plan B", the current nested resource provider model is designed primarily for stuff like NUMA nodes | |
| 15:10:22 | zhipeng_ | where you got this parent-child relationship | |
| 15:10:28 | crushil | So, how does that change our implementation? | |
| 15:10:42 | zhipeng_ | the Plan B extneds the scope to be more general, meaning for Cyborg use cases | |
| 15:11:05 | zhipeng_ | we could have multiple resource provider for each and every accelerators | |
| 15:11:14 | zhipeng_ | (if they deemed important for the workload) | |
| 15:11:23 | zhipeng_ | crushil the change is that | |
| 15:11:46 | zhipeng_ | our DB design has to align with the proposed nested resource provider/trait design | |
| 15:12:02 | zhipeng_ | at least DB schemas | |
| 15:12:18 | zhipeng_ | so that when cyborg agent populate our inventory to the placement api | |
| 15:12:25 | zhipeng_ | it could understand it correctly | |
| 15:13:59 | crushil | Ok, what about the other specs? | |
| 15:14:13 | zhipeng_ | not concerned that much :) | |
| 15:14:18 | crushil | gotcha | |
| 15:14:58 | zhipeng_ | So I'm thinking we might need two DB schemas | |
| 15:15:15 | zhipeng_ | the current one in the spec patch, could be used for the discovery phase | |
| 15:15:37 | zhipeng_ | that is when user start the cyborg service and then agent/driver do the discovery/pre-config | |
| 15:15:53 | zhipeng_ | collect what we have, on the host | |
| 15:16:16 | zhipeng_ | the second set of schema needs to be aligned with nested resource provider | |
| 15:16:33 | zhipeng_ | to interact with placement api and eventually nova-scheduler | |
| 15:16:53 | zhipeng_ | for the VM to select the correct accelerator resource | |
| 15:17:19 | jkilpatr | so we need to maintain two parallel db's for each purpose or do you mean we want to change the format in a future release? | |
| 15:18:20 | zhipeng_ | what I'm thinking is that we don't have exhaustive knowledge on the hardware now | |
| 15:18:58 | zhipeng_ | therefore we keep a seperate DB schema, the host side one should be more extendable or more abstract | |
| 15:19:17 | zhipeng_ | But on another thought | |
| 15:19:23 | zhipeng_ | it might be just too complex ..... | |
| 15:19:26 | zhipeng_ | what do you guys think | |
| 15:19:49 | jkilpatr | I think we should try and keep one db as much as possible, I don't want to try and maintain parallel sets of data | |
| 15:19:58 | zhipeng_ | that makes sense | |
| 15:20:24 | crushil | I agree, having multiple DBs is just clunky | |
| 15:21:04 | zhipeng_ | in that case we will just use the resource provider schema,I will follow up with Chris to see which one I should use | |
| 15:21:11 | zhipeng_ | the current one or the proposed one | |
| 15:22:19 | jkilpatr | sounds good. | |
| 15:22:56 | jkilpatr | Anything else on that subject? | |
| 15:22:59 | zhipeng_ | nope | |
| 15:23:13 | zhipeng_ | anything else from you guys on the open spec ? | |
| 15:23:33 | crushil | nope | |
| 15:23:51 | zhipeng_ | great | |
| 15:23:58 | zhipeng_ | #topic initial code development | |
| 15:24:05 | zhipeng_ | so, any roadblocks | |
| 15:24:39 | jkilpatr | been trying to understand oslo rpc and message passing and start structuring the conductor/agent | |
| 15:24:52 | zhipeng_ | sounds like a great start :) | |
| 15:25:02 | crushil | I have created stubs and I will push them up by the end of the week | |
| 15:25:11 | zhipeng_ | great ! | |
| 15:25:18 | jkilpatr | crushil, sounds good. | |
| 15:26:03 | zhipeng_ | let's do small pieces like Justin suggested | |
| 15:26:09 | crushil | I will fill them out rebased on top of the API and agent patches | |
| 15:26:10 | jkilpatr | so a lot of what we will be doing involves rpc between different components, so people with integrating parts need to talk to each other about interfaces | |
| 15:26:19 | jkilpatr | I don't think we should be too worried about a stable internal interface | |
| 15:26:29 | zhipeng_ | yes I agree | |
| 15:27:04 | zhipeng_ | oslo.messaging could provide everything we need | |
| 15:27:46 | jkilpatr | well sometimes we need rpc for example the driver should be called by the agent over rpc I'm thinking (we could invoke directly but I'm not sure if I want to do that) | |
| 15:29:38 | zhipeng_ | i think it should be done over rpc | |
| 15:29:57 | zhipeng_ | unless, we gave driver restful apis ? | |
| 15:30:31 | jkilpatr | I don't think that's the right application here. Our internal code needs to be more tightly integrated than restfulness allows. | |
| 15:30:45 | zhipeng_ | yep | |
| 15:31:07 | zhipeng_ | so rpc should be fine here | |
| 15:31:27 | zhipeng_ | i think at the moment, it is agent talking to the generic driver | |
| 15:31:50 | zhipeng_ | later on, we should design something like the neutron ml2 driver interface | |
| 15:32:29 | zhipeng_ | that every driver, vendor or not, implements the interface which rpc calls will go through | |
| 15:32:34 | zhipeng_ | in a rather standard way | |
| 15:33:29 | rushil | Ok. So, are we going to follow the neutron model vs the nova/cinder model? | |
| 15:33:47 | zhipeng_ | i think more like the neutron moddel | |
| 15:33:55 | zhipeng_ | for out-of-tree drivers | |
| 15:34:03 | rushil | But isn't that too complicated | |
| 15:34:09 | zhipeng_ | cinder and nova are mostly in-tree maintained drivers | |
| 15:34:20 | zhipeng_ | it won't be too complicated for us i think | |
| 15:34:40 | zhipeng_ | neutron is complicated because they have to define the type drivers and mechanism drivers | |
| 15:34:42 | rushil | Well, cinder has out of tree drivers based on whether you have CI or not | |
| 15:35:04 | zhipeng_ | I think in-tree drivers also requires the CI | |
| 15:35:14 | zhipeng_ | otherwise the cinder team removes your driver | |
| 15:35:38 | rushil | No, they just make it unsupported i.e. move it out of tree | |
| 15:35:54 | zhipeng_ | for us, as long as it is PCIe communicated devices, the driver interface won't be too complicated | |
| 15:36:12 | zhipeng_ | but if we need to support extra protocols, that is where things will get wild | |
| 15:36:24 | zhipeng_ | rushil ah okey | |
| 15:36:25 | rushil | Ok. I just want to make sure we don't make things more complicated than they should be | |
| 15:36:37 | zhipeng_ | yes that is always our goal | |
| 15:36:38 | jkilpatr | I can agree on a standard rpc interface but that's less complicated than I think you are making it out to be. | |
| 15:36:49 | zhipeng_ | we even wanted to skip the conductor :P | |
| 15:37:14 | jkilpatr | and I nearly got away with it too! | |
| 15:37:21 | zhipeng_ | jkilpatr haha | |
| 15:37:40 | rushil | Lol | |
| 15:39:26 | zhipeng_ | rushil the cyborg ml2 driver would be modeled from your generic driver implementation :P | |
| 15:40:37 | rushil | I wouldn't call it ml2 driver though | |
| 15:40:52 | zhipeng_ | of course we will have another name for it | |
| 15:41:14 | zhipeng_ | aluminum drivers :P | |
| 15:41:24 | zhipeng_ | for cyborg robots | |
| 15:41:45 | rushil | Hehe | |
| 15:42:36 | jkilpatr | Anyways I'll try have a stub up this week (conductor) and then agent next week. | |
| 15:42:46 | jkilpatr | depends on how other tasks go for me. | |
| 15:43:44 | rushil | jkilpatr: Cool | |
| 15:43:50 | zhipeng_ | sounds great, i got another colleague working on cyborg this week, so api code will be developed in parallel | |
| 15:44:09 | rushil | Awesome | |
| 15:44:10 | zhipeng_ | hopefully when we settled the spec, the initial code will come out | |