Posted	Nick	Remark
#openstack-cyborg - 2018-08-01
14:26:31	Li_Liu	shaohe_feng, that's great :)
14:27:00	Sundar	Shaohe: are you referring to DB fields?
14:27:43	shaohe_feng	So we a user request a accelerator with the function_uuid, cyborg filter the attribute of deployable.
14:28:08	shaohe_feng	Sundar, I did it in my poc.
14:29:06	shaohe_feng	Sundar, but currently, there is no attribute in the the driver return data info
14:29:18	shaohe_feng	so we need to improvement it.
14:30:50	shaohe_feng	Li_Liu's attribute DB is important to match our expect accelerators.
14:31:45	Sundar	The spec calls for function/region IDs etc. but it has not been implemented yet in the upstream code. You are saying that you have added function ID to the POC. Great.
14:32:10	shaohe_feng	Sundar, yes. You can have a loot.
14:34:02	shaohe_feng	Sundar, if we use you VAN.
14:34:37	shaohe_feng	Have consider how to report traits/resource class info?
14:35:53	shaohe_feng	and several interfaces, such as "def traits_report()" and "def resource_class_update"
14:35:59	Sundar	Overall, we need to take a look at the whole flow starting with discovery/enumeration. Hope you all are following the conversation between Eric and me in openstack-dev, IRC, etc.
14:36:12	Sundar	TShaohe, the driver/agent API spec talks about it
14:36:29	shaohe_feng	Great.
14:36:43	Sundar	The driver returns a structure (which should become an OVO), and the agent cna build RPs and traits out of that
14:37:24	shaohe_feng	what's field in the OVO?
14:37:26	xinran__	what is OVO
14:37:37	Sundar	I thoight about having the driver return traits directly and have the agent validate that, but that would probably make t flexible if we need to add new traits or modify existing ones
14:37:52	Sundar	OVO == oslo versioned objects
14:38:34	shaohe_feng	Sundar, You spec define well about different traits types.
14:38:44	Sundar	In simple terms, OVO is a Python dictionary with well=defined schema, a version and a namespace
14:38:54	shaohe_feng	I follow it in the POC
14:39:07	shaohe_feng	but not define how to report it.
14:39:39	Sundar	#lonk https://git.openstack.org/cgit/openstack/cyborg/tree/doc/specs/rocky/cyborg-agent-driver-api.rst?h=refs/changes/49/561849/7#n125
14:40:01	Sundar	Shaohe: Line 125 talks about function IDs and how the agent handles them
14:40:42	Sundar	Line 99 says what the field should look like
14:40:51	shaohe_feng	yes, I did return this format traits.
14:41:46	Sundar	Ah, typo: #link https://git.openstack.org/cgit/openstack/cyborg/tree/doc/specs/rocky/cyborg-agent-driver-api.rst?h=refs/changes/49/561849/7#n125
14:42:08	shaohe_feng	But I find it is not good for agent to report the traits.
14:42:32	shaohe_feng	For example, every devices may have different provider layers.
14:43:03	Sundar	Yes, the spec does not talk about ho the agent reports the info to Cyborg conductor or placement, because it is a driver/agent API spec
14:44:14	Sundar	"different provider layers" -- can you expand on that?
14:46:29	Li_Liu	I guess it refers to different layers of pf/vfs?
14:48:08	shaohe_feng	we list the the traits find there is a ``CUSTOM_<type>_<vendor>_FUNCTION_<function-uuid>``
14:48:50	wangzhh	Hi all. Sorry, I'm late...
14:49:13	shaohe_feng	such as CUSTOM_FPGA_INTEL_FUNCTION_1234, CUSTOM_FPGA_INTEL_FUNCTION_5678
14:49:41	shaohe_feng	Sundar, how Dow I know which traits is my expect?
14:50:08	shaohe_feng	I use a temporary solution.
14:50:33	shaohe_feng	I add a uuid/name map in yaml config.
14:51:08	shaohe_feng	such as 1234:crypto, 5678:ipsec
14:51:26	shaohe_feng	1234:crypto_v1.2
14:51:56	shaohe_feng	it support dynamically update.
14:52:27	shaohe_feng	I also report CUSTOM_FPGA_INTEL_FUNCTION_CRYPTO_V1_2 to the placement.
14:52:38	Li_Liu	shaohe_feng, you mean when the agent tries to update the traits, it has to know which ones to update?
14:52:46	Sundar	"how Do I know which traits is my expect" -- If the driver reports a function attribute (it is an optional field), the agent should build a function trait. Same for region type etc.
14:54:36	shaohe_feng	so a user list all traits, such as this cmd:
14:54:37	shaohe_feng	curl -g -i http://localhost/placement/traits \
14:54:37	shaohe_feng	-H "X-Auth-Token: $(openstack token issue -f value -c id)" \
14:54:37	shaohe_feng	-H "Content-Type: application/json" -H "Accept: application/json" \
14:54:37	shaohe_feng	-H "OpenStack-API-version: placement 1.26" |grep -oh '\w*CUSTOM*\w*'
14:54:50	Sundar	If you are asking how would we know whether a function trait has already been reported to Placement, that is a bit of a design/implementation detail. If you think the spec should address such aspects too, sure, we can add that
14:55:12	shaohe_feng	He can get both CUSTOM_FPGA_INTEL_FUNCTION_CRYPTO_V1_2 and CUSTOM_FPGA_INTEL_FUNCTION_1234
14:55:26	shaohe_feng	the can any one of them.
14:56:04	shaohe_feng	Sundar, I maintain this map in a config file.
14:56:17	shaohe_feng	not sure we can maintain them DB.
14:56:25	efried	In Dublin, for Nova, we said the user ought to be allowed to add and remove traits on providers as long as the traits aren't "owned" by Nova itself. If you wish to follow the same policy for Cyborg-owned providers, the above becomes relevant.
14:56:52	shaohe_feng	Li_Liu, Sundar, what's your suggestion?
14:57:15	Li_Liu	efried, can we specify Cyborg-owned providers?
14:57:36	efried	that would be whatever ones cyborg is in charge of creating.
14:57:42	Sundar	UUID/name maps etc. need further discussion and should be codified in specs. Who adds it and maintains it? I intentionally stayed away from such things, even if they are good to have, till the basics stabilize
14:58:27	shaohe_feng	Li_Liu, very Cyborg driver will maintain a sub provider by itself.
14:58:33	efried	I should note that it will be important for platform-specific drivers to have the ability to specify traits, presumably somewhere in the discovery/enumeration phase.
14:58:35	Sundar	It is good to have named instead of UUIDs - but that raises lots of issues about who defines it etc
14:59:23	Sundar	efried: Apart from the traits already defined in Cyborg specs, are you saying additional ad hoc traits would be needed?
14:59:51	Li_Liu	my suggestion is these can be stored the attribute table. it basically contains name, value, and uuid. Then map them to placement's traits
15:01:37	efried	Sundar: Yes. Have a look at
15:01:37	efried	#link Spec for nova-powervm device passthrough https://review.openstack.org/#/c/579359/10/doc/source/specs/rocky/device-passthrough.rst
15:01:37	efried	specifically around L220 where we enumerate traits we will generate based on device info; and L163 where we define how the user will be able to specify "ad hoc" traits via config file.
15:02:12	efried	This is how we're doing things while we wait for cyborg to become a thing :) And we'll need the same capabilities when that time comes.
15:03:07	shaohe_feng	Sundar, yes, so you think a good way to main the UUID/NAME map.
15:04:49	shaohe_feng	撒泼，
15:04:55	shaohe_feng	spam?
15:04:59	efried	yes
15:05:50	Sundar	efried: We wrote Cyborg specs on traits etc. and got them approved/merged after review. It would have been good to surface these requirements during the review period.
15:06:21	efried	True story.
15:06:36	efried	I should have been keeping better pace with those.
15:07:01	efried	My only excuse is having a lot on my plate.
15:07:20	Sundar	Don't mean to pick on you -- we probably have a systemic issue here. My concern is that, if we have ad hoc list of traits defined by each driver, it can be come unmanageable. It may be better to have a standardize dlist
15:07:48	shaohe_feng	efried, will you help to work on the nova side for nova/cyborg interaction?
15:08:27	Li_Liu	These traits can be vendor specific. When vendor add their drivers, they should comply with the standards
15:08:29	efried	Sundar: If "standardized" includes platform-specific traits specified by the platform-specific code in the driver, we can probably live with that.
15:08:56	xinran__	I agree with Li_Liu’s suggestion, and I have another question, do other devices like gpu need this kind of mapping?
15:09:09	efried	For example, the traits for DRC name & index are only going to exist on Power.
15:09:16	efried	shaohe_feng: Most likely, yes.
15:09:32	shaohe_feng	efried, good, Thanks.
15:09:53	efried	...and by the way, those traits are going to be dynamic.
15:10:14	efried	i.e. there's no way to predefine them
15:10:28	shaohe_feng	^ Sundar,
15:10:45	Sundar	efried: Basic question: why does Nova/placement need to know about DRC name/index etc.? Would you want to schedule instances to use specific GPU devices, as opposed to asking for *a* GPU of a specific type?
15:11:15	efried	Sundar: Yup, we at least need the *ability* to do that. Pets, not cattle.
15:11:33	shaohe_feng	efried, what does hoc mean?
15:11:53	efried	"ad hoc" means dynamic, non-predefined.
15:12:10	shaohe_feng	on the fly?
15:12:15	efried	Yes, exactly.
15:12:19	shaohe_feng	got it.
15:12:30	efried	I was going to use that exact phrase, but I thought throwing another idiom at you might not have worked :)
15:13:10	efried	Sundar: Also, the user-specified traits will be important when we're talking about devices with "a wire out the back". E.g. SR-IOV PF connected to a specific network, *somebody* needs to be able to specify a trait indicating the network.
15:13:27	efried	...so that the user can e.g. request a VF on a particular network.
15:13:47	efried	^ use case for truly ad hoc traits even in a "cattle" environment.

Earlier   Later