Posted	Nick	Remark
#openstack-cyborg - 2018-08-29
15:32:21	efried	Sundar: For right now, I'm interested in knowing how whitelisting is going to be done.
15:39:44	Sundar	Sure. Sorry, I got a call now, this will go on for 30 min.
15:39:50	Sundar	Please give me some time
15:54:32	efried	Sundar: I'll be leaving for a couple of hours soon. I'll try to reconnect with you here this afternoon if you're around; otherwise perhaps we can just have the discussion via the spec review.
16:00:26	kosamara	Sundar: I'll join here again tomorrow. Which timezone are you in? I'm in Europe.
16:05:17	Sundar	kosamara: I am in Pacific time zone. Is there something specific you want to talk about? We can agree on a time.
16:05:37	Sundar	efried: Sure, I'll catch you this afternoon
16:06:02	kosamara	Sundar: not yet, first I need to catch up on https://review.openstack.org/#/c/593726 and you on ours
16:06:52	kosamara	The basic question is if we should keep doing that spec for Nova or if we should contribute something related to that to Cyborg.
16:11:18	Sundar	kosamara: It was agreed at the rocky PTG that Cyborg is the way forward. I think device discovery should be in Cyborg.
16:12:00	Sundar	I need to catch up on https://review.openstack.org/#/c/591037/
16:17:16	Sundar	kosamara: Are you Konstantinos Samaras-Tsakiris, the author of the other spec https://review.openstack.org/#/c/591037/? :)
21:33:36	Sundar	efried: Please ping me when you can
#openstack-cyborg - 2018-08-30
02:32:47	openstackgerrit	wangzhh proposed openstack/cyborg master: Add "Report device data to cyborg" https://review.openstack.org/596691
02:58:17	openstackgerrit	wangzhh proposed openstack/cyborg master: Add "Report device data to cyborg" https://review.openstack.org/596691
03:15:21	openstackgerrit	wangzhh proposed openstack/cyborg master: Add "Report device data to cyborg" https://review.openstack.org/596691
05:43:33	openstackgerrit	Xinran WANG proposed openstack/cyborg master: Allocation/Deallocation API Specification https://review.openstack.org/597991
06:16:33	openstackgerrit	wangzhh proposed openstack/cyborg master: Add "Report device data to cyborg" https://review.openstack.org/596691
07:02:20	nguyenhai_	Is project openstack/os-acc belong to cyborg?
07:02:45	nguyenhai_	The core-reviewer of cyborg have responsible for openstack/os-acc or not? Thanks.
07:50:22	kosamara	Sundar: yes :)
07:51:25	kosamara	Sundar: But this spec is derived from efried's nova-powervm spec, which you have already commented on. The core principles are the same, but a lot of things have changed.
14:09:41	openstackgerrit	Merged openstack/os-acc master: import zuul job settings from project-config https://review.openstack.org/592836
14:16:17	openstackgerrit	Merged openstack/os-acc master: switch documentation job to new PTI https://review.openstack.org/592837
14:16:19	openstackgerrit	Merged openstack/os-acc master: add python 3.6 unit test job https://review.openstack.org/592838
14:19:12	openstackgerrit	Merged openstack/os-acc master: add lib-forward-testing-python3 test job https://review.openstack.org/592839
18:58:22	openstackgerrit	wangzhh proposed openstack/cyborg master: Add "Report device data to cyborg" https://review.openstack.org/596691
19:50:10	Sundar	efried: Please ping me when you can
19:50:19	efried	Sundar: Hi!
19:51:26	Sundar	Hi, I got tons of feedback on the os-acc spec ;). Can we discuss them now?
19:51:41	efried	okay, sure
19:51:47	efried	Though I think you got most of it from sean
19:52:48	Sundar	One feedback was that the attaching to the VM should be left to Nova virt driver, and os-acc should not do it. That is hypervisor-specific and that's what virt drivers are for.
19:52:55	Sundar	Do you agree with that?
19:53:30	efried	I... think so, yes. I think that's how neutron plugins operate. But maybe it's not how the os-vif model is set up.
19:54:20	efried	anyway, it's certainly hypervisor-specific. No question there.
19:54:35	efried	So when you say "os-acc" you really mean "the plugin at the behest of os-acc".
19:55:30	Sundar	One main difference between os-vif and os-acc that I am struggling to communicate is this: with os-vif, a VIF gets allocated, port binding happens and then the plug() operation happens.
19:57:08	Sundar	With os-acc, the equivalent of binding and plug cannot be separated cleanly. Example: A GPU may need to be reconfigured to create vGPUs of a certain type. Until that point, the accelerators of the right type don't even exist. So, the device may need to be configured to create the right accelerators, or change the inventory of accelerators, before we get the attach handle
19:58:40	efried	okay. That stuff is *also* hypervisor-specific.
19:59:09	Sundar	Sure, it is also device-specific. The device driver has to do this in a hypervisor-specific way
19:59:18	efried	May be the purview of the "driver" rather than the "plugin" but still.
19:59:44	Sundar	Yes, I think we already agree that the driver would be hypervisor-specific
19:59:51	efried	it may in fact be device *type* specific, not necessarily device specific.
20:00:02	efried	Anyway, I think we're in violent agreement on the high points here.
20:00:36	Sundar	Well, the actual act of writing to the device and manipulating it to create a new vGPU type or program a bitstream would in fcat be specific to device models or vendors.
20:02:19	Sundar	So, the Cyborg and its driver(s) would have to handle the equivalent of port binding (device specific) and plug. That gives us a VAN with the device end configured, ready to be attached to an instance
20:02:46	Sundar	After that, Nova virt can take that VAN and do the needful for that hypervisor, in a device-independen way
20:02:48	efried	and does that happen in a separate step prior to spawn?
20:03:18	Sundar	Yes. I shared a flow diagram with you this morning.
20:03:31	Sundar	Before we get to that :),
20:04:11	Sundar	the role of os-acc plugin is highly diminished with these aspects considered. The driver does the device end, Nova virt does the instance end
20:04:48	efried	"nova virt" by invoking the plugin through os-acc?
20:04:55	Sundar	The possible role for os-acc extensions is to handle device-compute interactions, such as NUMA affinity for interrupt vectors.
20:06:41	Sundar	Nova virt calls os-acc, and that in turn calls Cyborg in some way. When that call returns, Nova virt has to persist the instance - VAN association in Nova's db. neither os-acc nor Cyborg can do that, so it has to go to Nova virt. The subsequent step of attaching to the VM is already in Nova virt.
20:06:48	Sundar	So, what does the plugin do for attach?
20:07:07	efried	no idea
20:07:19	efried	if it's nothing, it's nothing.
20:08:03	Sundar	I understand your need for keeping things hypervisor-specific, but Nova virt is already hypervisor-specific, right?
20:08:14	efried	yup
20:08:39	efried	I don't really have a stake in getting lots of code into these plugins.
20:08:48	efried	I just want to make sure we don't end up with linux-isms in common code paths.
20:08:54	efried	linux/libvirt
20:09:16	Sundar	OK. Could you look at the flow diagram I shared this morning?
20:09:26	efried	If I didn't have to write a separate driver/plugin at all, I would be pretty happy. But I'm not sure that is going to happen.
20:10:08	Sundar	Not to spoil your afternoon, but I think you would need PowerVM drivers for GPUs, FPGAs, or whatever you support :)
20:11:23	Sundar	Are you looking at Power+KVM hypervisor too?
20:12:10	Sundar	brb in 5 min
20:12:19	efried	I don't know anything about pkvm
20:14:00	Sundar	back. ok
20:15:43	Sundar	Back to os-acc plugins. The possible role for os-acc plugins is to handle device-compute interactions, such as NUMA affinity for interrupt vectors. But it may be too much to bring them in this spec. Shall we put os-acc plugins as a placeholder until we get to that?
20:18:26	efried	"NUMA affinity for interrupt vectors" <== Greek to me.
20:18:51	Sundar	OK, NUMA affinity for devices in general?
20:19:11	efried	If you don't have an actual use case for os-acc plugins, then...
20:19:35	efried	NUMA affinity is something we want to be able to handle via resource provider structure.
20:19:48	efried	we can't yet
20:19:56	efried	but it's something we should bring up (again) in Denver.
20:20:49	Sundar	Sure. Shall we have a f2f chat at Denver? It'll be good to sync up and drive the specs to a close shortly thereafter
20:21:14	efried	definitely
20:21:30	efried	If I were you, I would ask for a nova/cyborg cross-project session.
20:21:52	efried	Based on what I know of people's schedules, I would say Tuesday would be a good day for it.
20:22:03	Sundar	I have entered a slot in Nova etherpad. May be I should ping melwitt?
20:23:02	efried	It might be better to schedule something during cyborg's time (M/T) where you can get the "placement cores" to visit the cyborg room and spend a couple of hours hashing stuff out.
20:23:33	efried	And keep it on the nova schedule in case we a) don't get everything sorted; and/or b) need a wider audience.
20:23:48	Sundar	OK
20:24:04	efried	Send something to the dev ML, like what Blazar did, to organize the Tuesday thing.
20:24:05	efried	oh
20:24:14	efried	Sorry, I forgot, Blazar sent that directly to people.
20:24:18	efried	not to the ML.
20:24:34	Sundar	Meanwhile, please LMK if you are ok with the flow diagram. Did you get it?
20:26:34	efried	I got it.
20:27:47	Sundar	Great. I'll grab lunch and get back in 15 min. We can shake it out after that.
20:29:57	efried	give me 20
21:07:14	Sundar	efried: Ready any time
21:08:00	efried	Sundar: Nova meeting in progress. Not sure how long it will last, but not beyond top of the hour. Will you be around?
21:09:31	Sundar	Yes
21:21:08	efried	Sundar: ō/
21:23:57	Sundar	That was quick.
21:24:32	efried	now where did I put that diagram...
21:24:56	efried	got it

Earlier   Later