| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-cyborg - 2018-08-01 | |||
| 14:51:56 | shaohe_feng | it support dynamically update. | |
| 14:52:27 | shaohe_feng | I also report CUSTOM_FPGA_INTEL_FUNCTION_CRYPTO_V1_2 to the placement. | |
| 14:52:38 | Li_Liu | shaohe_feng, you mean when the agent tries to update the traits, it has to know which ones to update? | |
| 14:52:46 | Sundar | "how Do I know which traits is my expect" -- If the driver reports a function attribute (it is an optional field), the agent should build a function trait. Same for region type etc. | |
| 14:54:36 | shaohe_feng | so a user list all traits, such as this cmd: | |
| 14:54:37 | shaohe_feng | curl -g -i http://localhost/placement/traits \ | |
| 14:54:37 | shaohe_feng | -H "X-Auth-Token: $(openstack token issue -f value -c id)" \ | |
| 14:54:37 | shaohe_feng | -H "Content-Type: application/json" -H "Accept: application/json" \ | |
| 14:54:37 | shaohe_feng | -H "OpenStack-API-version: placement 1.26" |grep -oh '\w*CUSTOM*\w*' | |
| 14:54:50 | Sundar | If you are asking how would we know whether a function trait has already been reported to Placement, that is a bit of a design/implementation detail. If you think the spec should address such aspects too, sure, we can add that | |
| 14:55:12 | shaohe_feng | He can get both CUSTOM_FPGA_INTEL_FUNCTION_CRYPTO_V1_2 and CUSTOM_FPGA_INTEL_FUNCTION_1234 | |
| 14:55:26 | shaohe_feng | the can any one of them. | |
| 14:56:04 | shaohe_feng | Sundar, I maintain this map in a config file. | |
| 14:56:17 | shaohe_feng | not sure we can maintain them DB. | |
| 14:56:25 | efried | In Dublin, for Nova, we said the user ought to be allowed to add and remove traits on providers as long as the traits aren't "owned" by Nova itself. If you wish to follow the same policy for Cyborg-owned providers, the above becomes relevant. | |
| 14:56:52 | shaohe_feng | Li_Liu, Sundar, what's your suggestion? | |
| 14:57:15 | Li_Liu | efried, can we specify Cyborg-owned providers? | |
| 14:57:36 | efried | that would be whatever ones cyborg is in charge of creating. | |
| 14:57:42 | Sundar | UUID/name maps etc. need further discussion and should be codified in specs. Who adds it and maintains it? I intentionally stayed away from such things, even if they are good to have, till the basics stabilize | |
| 14:58:27 | shaohe_feng | Li_Liu, very Cyborg driver will maintain a sub provider by itself. | |
| 14:58:33 | efried | I should note that it will be important for platform-specific drivers to have the ability to specify traits, presumably somewhere in the discovery/enumeration phase. | |
| 14:58:35 | Sundar | It is good to have named instead of UUIDs - but that raises lots of issues about who defines it etc | |
| 14:59:23 | Sundar | efried: Apart from the traits already defined in Cyborg specs, are you saying additional ad hoc traits would be needed? | |
| 14:59:51 | Li_Liu | my suggestion is these can be stored the attribute table. it basically contains name, value, and uuid. Then map them to placement's traits | |
| 15:01:37 | efried | Sundar: Yes. Have a look at | |
| 15:01:37 | efried | #link Spec for nova-powervm device passthrough https://review.openstack.org/#/c/579359/10/doc/source/specs/rocky/device-passthrough.rst | |
| 15:01:37 | efried | specifically around L220 where we enumerate traits we will generate based on device info; and L163 where we define how the user will be able to specify "ad hoc" traits via config file. | |
| 15:02:12 | efried | This is how we're doing things while we wait for cyborg to become a thing :) And we'll need the same capabilities when that time comes. | |
| 15:03:07 | shaohe_feng | Sundar, yes, so you think a good way to main the UUID/NAME map. | |
| 15:04:49 | shaohe_feng | 撒泼， | |
| 15:04:55 | shaohe_feng | spam? | |
| 15:04:59 | efried | yes | |
| 15:05:50 | Sundar | efried: We wrote Cyborg specs on traits etc. and got them approved/merged after review. It would have been good to surface these requirements during the review period. | |
| 15:06:21 | efried | True story. | |
| 15:06:36 | efried | I should have been keeping better pace with those. | |
| 15:07:01 | efried | My only excuse is having a lot on my plate. | |
| 15:07:20 | Sundar | Don't mean to pick on you -- we probably have a systemic issue here. My concern is that, if we have ad hoc list of traits defined by each driver, it can be come unmanageable. It may be better to have a standardize dlist | |
| 15:07:48 | shaohe_feng | efried, will you help to work on the nova side for nova/cyborg interaction? | |
| 15:08:27 | Li_Liu | These traits can be vendor specific. When vendor add their drivers, they should comply with the standards | |
| 15:08:29 | efried | Sundar: If "standardized" includes platform-specific traits specified by the platform-specific code in the driver, we can probably live with that. | |
| 15:08:56 | xinran__ | I agree with Li_Liu’s suggestion, and I have another question, do other devices like gpu need this kind of mapping? | |
| 15:09:09 | efried | For example, the traits for DRC name & index are only going to exist on Power. | |
| 15:09:16 | efried | shaohe_feng: Most likely, yes. | |
| 15:09:32 | shaohe_feng | efried, good, Thanks. | |
| 15:09:53 | efried | ...and by the way, those traits are going to be dynamic. | |
| 15:10:14 | efried | i.e. there's no way to predefine them | |
| 15:10:28 | shaohe_feng | ^ Sundar, | |
| 15:10:45 | Sundar | efried: Basic question: why does Nova/placement need to know about DRC name/index etc.? Would you want to schedule instances to use specific GPU devices, as opposed to asking for *a* GPU of a specific type? | |
| 15:11:15 | efried | Sundar: Yup, we at least need the *ability* to do that. Pets, not cattle. | |
| 15:11:33 | shaohe_feng | efried, what does hoc mean? | |
| 15:11:53 | efried | "ad hoc" means dynamic, non-predefined. | |
| 15:12:10 | shaohe_feng | on the fly? | |
| 15:12:15 | efried | Yes, exactly. | |
| 15:12:19 | shaohe_feng | got it. | |
| 15:12:30 | efried | I was going to use that exact phrase, but I thought throwing another idiom at you might not have worked :) | |
| 15:13:10 | efried | Sundar: Also, the user-specified traits will be important when we're talking about devices with "a wire out the back". E.g. SR-IOV PF connected to a specific network, *somebody* needs to be able to specify a trait indicating the network. | |
| 15:13:27 | efried | ...so that the user can e.g. request a VF on a particular network. | |
| 15:13:47 | efried | ^ use case for truly ad hoc traits even in a "cattle" environment. | |
| 15:16:25 | Sundar | efried and all: A process question: how can we ensure that we have got all requirements and that the next round of spec updates shall be adequate to start implementation for Stein? We had a bunch of plans for Rocky and we want to ensure they are delivered in Stein. Open to ideas! | |
| 15:16:54 | efried | Sundar: If you solve that one, you get a Nobel. | |
| 15:18:04 | Li_Liu | rofl | |
| 15:18:06 | efried | But the neat thing about our process is that it doesn't have to be perfect and fully-formed the first time around. | |
| 15:18:29 | efried | You put out *something* and then improve on it incrementally. | |
| 15:18:47 | efried | Surely you didn't think you were going to be done after one release. | |
| 15:18:49 | Li_Liu | workable -> usable | |
| 15:20:31 | Sundar | :) Shall we define a date before Stein's last spec approval date, by which time we can review and approve the needed Cyborg specs, and get started on implementation after that? | |
| 15:21:23 | Sundar | If any stakeholder wants an update after that, we can address that in T release | |
| 15:22:14 | Sundar | Oct 22-26 is Stein Milestone 1. | |
| 15:22:24 | Sundar | That is 3 months away | |
| 15:22:51 | Sundar | I suspect we can get the specs done in 1 month if we put our collective minds to it. Comments? | |
| 15:23:56 | efried | Just to level set, a statement like ^ is true in isolation, but you can't guarantee that all the stakeholders will have the time to dedicate to it to make that timeline feasible. | |
| 15:24:09 | efried | That said, I will do what I can to carve out the time. | |
| 15:24:52 | Sundar | Thanks, Eric. | |
| 15:25:35 | Sundar | To elaborate, if we wait until Oct end, we get very little time for implementation given that Dec and 1st half Jan are usually in holiday mode | |
| 15:26:12 | Sundar | Anyways, I'll take an AR to read the spec you mentioned, and update the specs on traits etc. | |
| 15:26:30 | xinran__ | So did we have a solution about how to map function uuid to name, or other things like that? | |
| 15:27:16 | Sundar | On UUID/name maps, whether that should come from the bitstream developer or the operator is an open question. The UUID itself comes the bitstream developer | |
| 15:28:01 | Sundar | If the driver is going to define traits, that is another wrinkle because the driver comes from the device vendor, which is yet another role | |
| 15:28:50 | Sundar | So, we need to have some input on how people use this in practice -- feedback needed from operators, bitstream developers etc. | |
| 15:29:39 | Sundar | I think this needs to be solved -- we should go to names, but any premature move before people understand and use Cyborg may require fixing later -- and that can be messy with upgrade issues wtc | |
| 15:32:21 | Sundar | Can we stick to UUIDs for now, and take up the name translation in subsequent discussion? That is the lowest common denominator anyway. | |
| 15:33:40 | efried | sorry, uuids/names for what specifically? | |
| 15:33:41 | xinran__ | Is it possible to translate the uuid to name in cyborg according to the data structure got by driver? | |
| 15:33:48 | Li_Liu | sure, that requires less modifications in the future I think | |
| 15:35:07 | Li_Liu | xinran__, those uuid are reported by drivers, Cyborg needs to track them | |
| 15:35:11 | Sundar | Xinran: as I explained, we have to ask should define function names. The driver comes from a device vendor (like Intel/Xilinx) -- they may not know anything about a specific function developed by a third party | |
| 15:35:14 | Li_Liu | Sundar, am i right? | |
| 15:36:08 | Sundar | Li_Liu: the UUIDs are reported by drivers, based on what they discover from the device itself. | |
| 15:36:20 | Sundar | However, where does the name came from? | |
| 15:36:38 | xinran__ | Yes that’s the problem | |
| 15:36:54 | Li_Liu | I believe name should also reported by driver | |
| 15:37:31 | Li_Liu | driver should know the name, and report them to Cyborg | |
| 15:37:47 | Sundar | There are two options: have the bitstream developer define an optional name (the current metadata spec already provides for it), or have the cloud operator define a UUID/name map like Shaohe mentioned | |
| 15:37:52 | xinran__ | How does driver know the name? | |
| 15:37:56 | Sundar | Both have their pros and cons. | |
| 15:39:27 | Sundar | The driver does not know any function names in the general case. If the driver itself is involved in programming a bitstream and the bitstream metadata has a name, well and good. But, the device may be pre-programmed or the bitstream many not have that metadata | |
| 15:39:44 | Li_Liu | driver gets the name from either: 1. device information read from hardware. 2. When loading a bitstream, driver can decode the name from the image file | |
| 15:39:52 | dolpher | sundar: operator doest define name, they should got name from vendor, and put them into the mapping file | |
| 15:40:28 | dolpher | so agent or driver can read it then update to trait | |
| 15:40:44 | Sundar | Li_Liu: Device info from hardware does not include function names today, for any major vendor AFAIK | |