| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2019-03-13 | |||
| 03:25:01 | Sundar | We are not reporting anything ti Placement yet, right? | |
| 03:25:04 | Coco_gao | xinranwang, that's the problem when across nodes, name maybe same right? | |
| 03:25:07 | Li_Liu | how about when we report the deployable to placement API, we concate name+uuid | |
| 03:25:53 | Sundar | Li_Liu: good idea. I'll get back with the name convention for nested RPs | |
| 03:26:16 | wangzhh | xinranwang, what if different node has same device? Is it unique? | |
| 03:26:33 | xinranwang | if we support NRP, we can identify which host the deployable locate, should it be ok to have same deployable name in different compute node ? | |
| 03:27:13 | Coco_gao | xinranwang, that will be ok, i think. | |
| 03:27:33 | shaohe_feng_ | the fpga devices name is generated by the kernel. | |
| 03:27:37 | wangzhh | xinranwang, Not really, Now it is global unique. | |
| 03:27:43 | shaohe_feng_ | the name is unique | |
| 03:28:33 | shaohe_feng_ | it does not mater if different node has same device | |
| 03:28:40 | Coco_gao | the reason why we need to keep unique from the aspect of driver ovo is that we need to identify the deployable. But driver ovo is compared in the same node, so, the name need only to be unique in one host. | |
| 03:29:06 | shaohe_feng_ | Coco_gao: yes. | |
| 03:29:33 | wangzhh | Coco, so we should change db, it is global unique now. | |
| 03:29:33 | shaohe_feng_ | for device@host is unique | |
| 03:29:39 | xinranwang | so i think it's ok to have same deployable name in different compute node, in placement side. But name should be unique on same compute node. | |
| 03:30:17 | shaohe_feng_ | the name is not used to identify a device | |
| 03:30:26 | Coco_gao | wangzhh, Sundar and all, maybe we need to change the db constrains on the deployable table, name field. | |
| 03:30:46 | Coco_gao | do you argree if I modify that? | |
| 03:31:03 | Li_Liu | what constrain? | |
| 03:31:07 | shaohe_feng_ | just a Prompt for human | |
| 03:31:21 | Coco_gao | the name field is unique in deployable table. | |
| 03:31:33 | Li_Liu | ah, ok | |
| 03:31:42 | Li_Liu | go ahead | |
| 03:31:50 | Li_Liu | no problem on my side | |
| 03:31:55 | shaohe_feng_ | I agree | |
| 03:31:58 | Coco_gao | OK, thank you are for the advice. | |
| 03:32:05 | Coco_gao | all | |
| 03:32:07 | wangzhh | Of course. But how to handle device like gpu, <device_name>_<address>? | |
| 03:32:24 | Sundar | Coco_gao: I think it is ok to make it unique because: there is some proposed convention to name nested RPs like '<hostname>_<numaNode>_<x>' and x must be unique within a node anyway for us. | |
| 03:32:25 | shaohe_feng_ | just keep id/uuid unique. it it machine readable. | |
| 03:33:13 | shaohe_feng_ | unique in a node is ok. | |
| 03:33:18 | wangzhh | shaohe, when driver report a device, it does not have a uuid. | |
| 03:33:31 | shaohe_feng_ | not need global | |
| 03:33:49 | shaohe_feng_ | wangzhh: agent gen one for it. :) | |
| 03:34:05 | shaohe_feng_ | bus is also unique. | |
| 03:34:17 | shaohe_feng_ | bus is also machine readable. | |
| 03:34:30 | Coco_gao | Sundar, the problem is how to generate x to make sure same card is using the same x when reporting. | |
| 03:34:56 | wangzhh | shaohe, agent will generate the uuid every time? | |
| 03:35:08 | shaohe_feng_ | wangzhh: no. just once. | |
| 03:35:16 | shaohe_feng_ | wangzhh: it need to check the bus. | |
| 03:35:35 | shaohe_feng_ | wangzhh: on a node, bus is used for machine read . | |
| 03:35:51 | shaohe_feng_ | on a cluster, uuid is used for machine read | |
| 03:36:06 | Sundar | There may not be a PCI bdf in all hypervisors. | |
| 03:36:09 | wangzhh | shaohe, I suppose you mean to generate it at first time. | |
| 03:36:23 | shaohe_feng_ | Coco_gao: the x can be generated by the bus. | |
| 03:36:33 | shaohe_feng_ | Coco_gao: let me show you an example | |
| 03:36:37 | shaohe_feng_ | wangzhh: yes. | |
| 03:36:44 | Coco_gao | thanks shaohe | |
| 03:36:51 | Li_Liu | if there's no bdf, can we use uuid? | |
| 03:37:35 | shaohe_feng_ | Li_Liu: there's another identification without bdf | |
| 03:37:36 | shaohe_feng_ | for | |
| 03:37:38 | wangzhh | But agent doesn't know which time it is. | |
| 03:37:51 | wangzhh | shaohe_feng_ | |
| 03:38:13 | shaohe_feng_ | wangzhh: it need to check. if the bus not in the db, then it is the first time. | |
| 03:38:14 | Li_Liu | shaohe_feng_, sure that also works | |
| 03:38:38 | shaohe_feng_ | seems mdev has a uuid. | |
| 03:38:48 | shaohe_feng_ | and usb has it own bus. | |
| 03:38:53 | Sundar | The driver should report a unique id within the node for each device. It could be PCI bdf for libvirt or whatever is unique for PowerVM and others | |
| 03:38:54 | wangzhh | If so, agent should query db first. do something like diff? | |
| 03:39:08 | Sundar | Then that could be the x factor | |
| 03:39:37 | Sundar | wangzhh: No, agent should not query db. For 2 reasons: scaling, upgrades can change db schema | |
| 03:39:46 | shaohe_feng_ | wangzhh: yes. wen agent start. ti should sync with db firstly | |
| 03:39:47 | wangzhh | +1 | |
| 03:39:50 | shaohe_feng_ | when | |
| 03:40:31 | wangzhh | shaohe_feng_ agent doesn't query db now. | |
| 03:40:33 | shaohe_feng_ | Sundar: no, it should sync when it start. and can keep the info in cache. | |
| 03:40:57 | Sundar | Agent should not keep state. Even if it reads db at startup, it cannot assume that it will remain in sync, because operator can update config | |
| 03:41:25 | Sundar | No cache, please. We will hit all kinds of issues with stale caches, aging, etc. | |
| 03:41:28 | Li_Liu | shaohe_feng_, is the cache only containing the information related to the node? | |
| 03:41:39 | shaohe_feng_ | yes. | |
| 03:41:41 | wangzhh | Agree with sundar at this part. :) | |
| 03:41:49 | shaohe_feng_ | it's own node info. | |
| 03:42:12 | shaohe_feng_ | let me show you what I do. | |
| 03:42:20 | Li_Liu | Sundar, I think it should be ok if it only holds its own information in cache | |
| 03:42:53 | wangzhh | shaohe_feng_ haha talk is cheap, show me your code. :) | |
| 03:42:54 | Sundar | Li_Liu: The operator may want to disable or enable specific devices, or do other config. | |
| 03:43:21 | shaohe_feng_ | wangzhh: yes, I do show you code | |
| 03:43:28 | Coco_gao | before diff, the agent should get the old driver ovo, is that from db or cache? | |
| 03:43:30 | wangzhh | Cool. | |
| 03:43:34 | shaohe_feng_ | wangzhh: I have implemented it. | |
| 03:43:43 | Sundar | Li_Liu: Then we have to propagate such changes to each agent, ensure that it has received it, etc. The agent doesn't need any state for discovery -- just add a unique field that driver reports. | |
| 03:43:53 | shaohe_feng_ | I report the placement by: device_name@host this is unique | |
| 03:44:07 | shaohe_feng_ | and I just pud the device_name in cyborg db | |
| 03:44:27 | shaohe_feng_ | it can works well, no any conflict, | |
| 03:44:28 | Coco_gao | I agree with shaohe. | |
| 03:44:51 | shaohe_feng_ | for placement use device_name@host for index. | |
| 03:45:02 | Sundar | Coco_gao: Again, there are some conventions proposed for nested RP names. I am still trying to find the spec/doc where I saw that. | |
| 03:45:06 | shaohe_feng_ | but cyborg does not use device_name for index. | |
| 03:46:54 | shaohe_feng_ | Sundar: that's 2 things, but if you want to keep it same. it is OK. | |
| 03:47:50 | Sundar | shaohe_feng: There's no point in making them different. The only reason why we have a deployable name is to report to placement | |
| 03:47:51 | shaohe_feng_ | the big problems it not this. | |
| 03:48:20 | Li_Liu | Sundar, please help to find out the conventions. shaohe_feng_, could you share you code with us? | |
| 03:48:37 | shaohe_feng_ | the big problems is enumeration. | |
| 03:48:41 | xinranwang | maybe keep deployable name unique on same compute, and add hostname like "@host" when report to placement. | |
| 03:48:45 | Li_Liu | It seems we need some further discussion on this issue, we can discuss it in tomorrow's zoom sync | |
| 03:48:55 | xinranwang | i believe that's what shaohe_feng_ did. | |
| 03:49:37 | shaohe_feng_ | Li_Liu: if restart, and some change on the host. the enumeration may change the bus of a same device. | |
| 03:50:05 | Li_Liu | shaohe_feng_, yea, I know | |
| 03:50:15 | shaohe_feng_ | I means cloud provider may resize the hardware on the host | |