| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2019-03-13 | |||
| 03:17:41 | Li_Liu | as least make it align with our docs and APIs | |
| 03:18:30 | Coco_gao | Hi Sundar, still one thing about driver ovo. Is deployable name unique? why is that? | |
| 03:18:35 | Coco_gao | Thanks a lot. | |
| 03:19:08 | Sundar | Coco_gao: I think so, because it will be used as resource provider name in Placement, and that must be unique AFAIK | |
| 03:19:43 | Li_Liu | How are we going to guarantee deployable's name's uniqueness | |
| 03:20:05 | Li_Liu | are we doing the check when the resource is reported? | |
| 03:20:25 | Coco_gao | we can't if we set the name field in the drivers. | |
| 03:20:33 | Sundar | Coco_gao: I don't see explicit documentation that it must be unique. I will check and get back. | |
| 03:21:31 | Sundar | Coco_gao and all: why can't Cyborg agent construct the name from other fields like vendor, type, etc., and add a unique id? | |
| 03:21:55 | Sundar | E.g. 'INTEL_FPGA_PAC_CARD_ID1' | |
| 03:22:08 | Li_Liu | is this ID1 a uuid? | |
| 03:22:26 | Sundar | Li_Liu: I was thinking a simple integer | |
| 03:22:38 | Sundar | Oh, wait | |
| 03:22:50 | Sundar | There is a convention for naming nested RPs | |
| 03:22:59 | Sundar | It is based on compute node name | |
| 03:23:07 | Sundar | I will check and send email. | |
| 03:23:22 | xinranwang | now the deployable name is the filename in /sys/class/fpga, it's unique | |
| 03:24:01 | Sundar | xinranwang: It is unique within a compute node | |
| 03:24:10 | Sundar | The same name can repeat across nodes | |
| 03:24:16 | Coco_gao | Sundar, I agree we'd better do that in agent. | |
| 03:25:01 | Sundar | We are not reporting anything ti Placement yet, right? | |
| 03:25:04 | Coco_gao | xinranwang, that's the problem when across nodes, name maybe same right? | |
| 03:25:07 | Li_Liu | how about when we report the deployable to placement API, we concate name+uuid | |
| 03:25:53 | Sundar | Li_Liu: good idea. I'll get back with the name convention for nested RPs | |
| 03:26:16 | wangzhh | xinranwang, what if different node has same device? Is it unique? | |
| 03:26:33 | xinranwang | if we support NRP, we can identify which host the deployable locate, should it be ok to have same deployable name in different compute node ? | |
| 03:27:13 | Coco_gao | xinranwang, that will be ok, i think. | |
| 03:27:33 | shaohe_feng_ | the fpga devices name is generated by the kernel. | |
| 03:27:37 | wangzhh | xinranwang, Not really, Now it is global unique. | |
| 03:27:43 | shaohe_feng_ | the name is unique | |
| 03:28:33 | shaohe_feng_ | it does not mater if different node has same device | |
| 03:28:40 | Coco_gao | the reason why we need to keep unique from the aspect of driver ovo is that we need to identify the deployable. But driver ovo is compared in the same node, so, the name need only to be unique in one host. | |
| 03:29:06 | shaohe_feng_ | Coco_gao: yes. | |
| 03:29:33 | shaohe_feng_ | for device@host is unique | |
| 03:29:33 | wangzhh | Coco, so we should change db, it is global unique now. | |
| 03:29:39 | xinranwang | so i think it's ok to have same deployable name in different compute node, in placement side. But name should be unique on same compute node. | |
| 03:30:17 | shaohe_feng_ | the name is not used to identify a device | |
| 03:30:26 | Coco_gao | wangzhh, Sundar and all, maybe we need to change the db constrains on the deployable table, name field. | |
| 03:30:46 | Coco_gao | do you argree if I modify that? | |
| 03:31:03 | Li_Liu | what constrain? | |
| 03:31:07 | shaohe_feng_ | just a Prompt for human | |
| 03:31:21 | Coco_gao | the name field is unique in deployable table. | |
| 03:31:33 | Li_Liu | ah, ok | |
| 03:31:42 | Li_Liu | go ahead | |
| 03:31:50 | Li_Liu | no problem on my side | |
| 03:31:55 | shaohe_feng_ | I agree | |
| 03:31:58 | Coco_gao | OK, thank you are for the advice. | |
| 03:32:05 | Coco_gao | all | |
| 03:32:07 | wangzhh | Of course. But how to handle device like gpu, <device_name>_<address>? | |
| 03:32:24 | Sundar | Coco_gao: I think it is ok to make it unique because: there is some proposed convention to name nested RPs like '<hostname>_<numaNode>_<x>' and x must be unique within a node anyway for us. | |
| 03:32:25 | shaohe_feng_ | just keep id/uuid unique. it it machine readable. | |
| 03:33:13 | shaohe_feng_ | unique in a node is ok. | |
| 03:33:18 | wangzhh | shaohe, when driver report a device, it does not have a uuid. | |
| 03:33:31 | shaohe_feng_ | not need global | |
| 03:33:49 | shaohe_feng_ | wangzhh: agent gen one for it. :) | |
| 03:34:05 | shaohe_feng_ | bus is also unique. | |
| 03:34:17 | shaohe_feng_ | bus is also machine readable. | |
| 03:34:30 | Coco_gao | Sundar, the problem is how to generate x to make sure same card is using the same x when reporting. | |
| 03:34:56 | wangzhh | shaohe, agent will generate the uuid every time? | |
| 03:35:08 | shaohe_feng_ | wangzhh: no. just once. | |
| 03:35:16 | shaohe_feng_ | wangzhh: it need to check the bus. | |
| 03:35:35 | shaohe_feng_ | wangzhh: on a node, bus is used for machine read . | |
| 03:35:51 | shaohe_feng_ | on a cluster, uuid is used for machine read | |
| 03:36:06 | Sundar | There may not be a PCI bdf in all hypervisors. | |
| 03:36:09 | wangzhh | shaohe, I suppose you mean to generate it at first time. | |
| 03:36:23 | shaohe_feng_ | Coco_gao: the x can be generated by the bus. | |
| 03:36:33 | shaohe_feng_ | Coco_gao: let me show you an example | |
| 03:36:37 | shaohe_feng_ | wangzhh: yes. | |
| 03:36:44 | Coco_gao | thanks shaohe | |
| 03:36:51 | Li_Liu | if there's no bdf, can we use uuid? | |
| 03:37:35 | shaohe_feng_ | Li_Liu: there's another identification without bdf | |
| 03:37:36 | shaohe_feng_ | for | |
| 03:37:38 | wangzhh | But agent doesn't know which time it is. | |
| 03:37:51 | wangzhh | shaohe_feng_ | |
| 03:38:13 | shaohe_feng_ | wangzhh: it need to check. if the bus not in the db, then it is the first time. | |
| 03:38:14 | Li_Liu | shaohe_feng_, sure that also works | |
| 03:38:38 | shaohe_feng_ | seems mdev has a uuid. | |
| 03:38:48 | shaohe_feng_ | and usb has it own bus. | |
| 03:38:53 | Sundar | The driver should report a unique id within the node for each device. It could be PCI bdf for libvirt or whatever is unique for PowerVM and others | |
| 03:38:54 | wangzhh | If so, agent should query db first. do something like diff? | |
| 03:39:08 | Sundar | Then that could be the x factor | |
| 03:39:37 | Sundar | wangzhh: No, agent should not query db. For 2 reasons: scaling, upgrades can change db schema | |
| 03:39:46 | shaohe_feng_ | wangzhh: yes. wen agent start. ti should sync with db firstly | |
| 03:39:47 | wangzhh | +1 | |
| 03:39:50 | shaohe_feng_ | when | |
| 03:40:31 | wangzhh | shaohe_feng_ agent doesn't query db now. | |
| 03:40:33 | shaohe_feng_ | Sundar: no, it should sync when it start. and can keep the info in cache. | |
| 03:40:57 | Sundar | Agent should not keep state. Even if it reads db at startup, it cannot assume that it will remain in sync, because operator can update config | |
| 03:41:25 | Sundar | No cache, please. We will hit all kinds of issues with stale caches, aging, etc. | |
| 03:41:28 | Li_Liu | shaohe_feng_, is the cache only containing the information related to the node? | |
| 03:41:39 | shaohe_feng_ | yes. | |
| 03:41:41 | wangzhh | Agree with sundar at this part. :) | |
| 03:41:49 | shaohe_feng_ | it's own node info. | |
| 03:42:12 | shaohe_feng_ | let me show you what I do. | |
| 03:42:20 | Li_Liu | Sundar, I think it should be ok if it only holds its own information in cache | |
| 03:42:53 | wangzhh | shaohe_feng_ haha talk is cheap, show me your code. :) | |
| 03:42:54 | Sundar | Li_Liu: The operator may want to disable or enable specific devices, or do other config. | |
| 03:43:21 | shaohe_feng_ | wangzhh: yes, I do show you code | |
| 03:43:28 | Coco_gao | before diff, the agent should get the old driver ovo, is that from db or cache? | |
| 03:43:30 | wangzhh | Cool. | |