Posted	Nick	Remark
#openstack-nova - 2018-02-27
09:56:37	efried	alex_xu: It depends to some extent on how the accelerators are modeled. If they're included in the compute node's provider tree, then yes, ?in_tree=<compute_uuid> on GET /allocation_candidates would be very appropriate.
09:57:18	alex_xu	efried: yea
09:57:33	efried	alex_xu: And/or if the accelerator is made part of an aggregate (not sure if that makes sense ever, but it extends to the more general model for e.g. bandwidth providers) then member_of
09:57:47	efried	The latter is a Rocky bp. The former is not (yet).
09:58:42	Kevin_Zheng	Any idea how Nova can consume resource reported by cyborg?
09:59:20	alex_xu	efried: I didn't get the accelerator is made part of an aggregate. The aggregate in Nova?
09:59:51	openstackgerrit	Chris Dent proposed openstack/nova master: WIP: Move resource provider objects into placement hierarchy https://review.openstack.org/540049
09:59:52	openstackgerrit	Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435
09:59:52	openstackgerrit	Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
10:00:05	alex_xu	Kevin_Zheng: depends on the direction we go I guess, with bauzas way, it can be done by cyborg API to return some resource class and traits. With the post-deploy attach, that is cyborg own thing
10:00:10	bauzas	folks, I'm in the Scientifig WG room so I can't follow the convo
10:00:21	efried	alex_xu: It probably doesn't make sense for accelerators to be modeled as true sharing providers.
10:00:23	alex_xu	bauzas: no worries
10:00:43	efried	alex_xu: Though be aware that sharing providers aren't the only possible use case for aggregates.
10:01:20	efried	alex_xu: In the case of accelerators, which are pieces of hardware physically attached to the host, it most likely makes sense for them to be somewhere in the compute node's provider tree.
10:02:25	efried	alex_xu: But for something like a bandwidth provider, where the component doing the work is maybe an external switch that manages traffic from multiple hosts, a sharing provider might be totally appropriate. (I need to reread the network bandwidth provider spec.)
10:02:26	alex_xu	efried: yea, I can't image how accelerators shared between hosts
10:02:35	gibi	Kevin_Zheng: as a wild alternative there is flavor extra_spec https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/custom-resource-classes-in-flavors.html
10:03:08	alex_xu	efried: ah, that sounds make sense for bindwidth
10:03:10	gibi	efried: correct, a TOR switch would be a shared provider
10:03:11	efried	gibi: Here's a not-so-secret secret - that's how PowerVM is planning to do accelerator attach.
10:03:24	efried	at least tentatively
10:03:35	bauzas	efried: how the FPGA accelerators are modeled looks to me a Cyborg-only discusion
10:03:51	efried	gibi: Though we may or may not need custom RCs.
10:03:54	gibi	efried: We had our internal plans to do bandwidth with flavor extra_spec but it got voted down internally
10:04:06	bauzas	I mean, I like the idea that the folks model their resources like they want and just allocate the resources like they want too
10:04:19	efried	bauzas: ++
10:04:23	Kevin_Zheng	gibi, alex_xu so cyborg is in charge of reporting accelerator resources, but how should it be connected to Nova Hosts?
10:04:43	efried	Kevin_Zheng: That's a great question.
10:04:50	bauzas	the only Nova piece would be the fact to tie the instance to the compute that has the accelerated resource and do the binding
10:05:03	gibi	bauzas: the allocation part getting tricky if we want to avoid races during boot. With post-attach it seems a lot easyer
10:05:11	melwitt	fyi we have some time scheduled to chat with the cyborg team at 2pm this afternoon at the cyborg room
10:05:11	efried	bauzas, Kevin_Zheng: Like attach_interface, but for device?
10:05:19	openstackgerrit	Stephen Finucane proposed openstack/nova master: Stop using mox in api/openstack/fakes.py https://review.openstack.org/325759
10:05:38	gibi	melwitt: yeah, I'm warming up for that now :)
10:05:44	bauzas	melwitt: gtk, I wasn't knowing that
10:05:56	bauzas	melwitt: I just spent 2 hours of my time already in that room :p
10:07:07	gibi	Kevin_Zheng: nova creates the compute RP and names it after the host name of the compute so if cyborg knows the host name where the accelerator is then cyborg can look up the compute RP created by nova in placement by the host name
10:07:33	alex_xu	gibi: ++
10:07:36	melwitt	oh, okay. on the dev ML last week or the week before we had a thread where we agreed on some time to discuss nova interaction things with the cyborg team
10:08:18	melwitt	apologies if I didn't communicate that well enough, I put it on our ptg etherpad agenda times last week when I added times
10:08:29	Kevin_Zheng	gibi hmm
10:08:36	bauzas	melwitt: my bad, I probably messed up reading that ;)
10:08:53	bauzas	melwitt: 2pm in their room ?
10:09:17	melwitt	yes
10:10:05	gibi	Kevin_Zheng: at least the host name will be the key information when neturon will create the networking RPs under the compute RP
10:10:20	gibi	Kevin_Zheng: so I think the same can be done in cyborg
10:10:42	mriedem	nova passes neutron the host name in the port binding so neutron can lookup the compute node RP yeah
10:10:43	Kevin_Zheng	gibi yeah, I can see the picture now
10:10:46	mriedem	for ironic, not so much...
10:12:13	melwitt	FYI on thursday we're gonna start nova/cinder at 9:15 instead of 9:00
10:12:33	alex_xu	gibi: I guess in the future, the accelerator device also has the NUMA affinity thing, so the cyborg also need to look up the NUMA rp
10:12:53	alex_xu	at that time, NUMA rp should have some naming rule also, i guess
10:14:02	mriedem	need another core on this https://review.openstack.org/#/c/526772/ - leaking passwords isn't good
10:15:34	gibi	mriedem: the network RP during the init of the neturon agent running on the given compute host, so that agent knows the host name without any nova involvement
10:16:01	gibi	mriedem: the network RP is created during...
10:16:01	mriedem	gibi: true
10:16:17	openstackgerrit	Chris Dent proposed openstack/nova master: [placement] use simple FaultWrapper https://review.openstack.org/533752
10:16:18	openstackgerrit	Chris Dent proposed openstack/nova master: WIP: Refactor WSGI apps and utils to limit imports https://review.openstack.org/533797
10:16:47	mriedem	lyarwood: tonyb: we have some stable/ocata changes that need another +2 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/ocata+label:Code-Review=2
10:17:42	gibi	mriedem: but yeah, if we talk about letting neutron create the bandwidth allocation in placement then the host name in the binding helps. However I'm more favor of duing the allocation from nova according to the selected allocation candidate
10:18:34	gibi	alex_xu: yeah, NUMA is a good question for me. We have to make NUMA RP named after some host level id that can be known by other agents
10:21:48	gibi	10:58 < frickler> infra-root: seeing bumpy response times from gerrit, increasing GC times, too. maybe
10:21:51	gibi	restart during PTG lunch time?
10:22:07	gibi	alex_xu: so gerrit is slow for everybody
10:22:46	jhesketh	gibi: it's being restarted now
10:24:03	Kevin_Zheng	So I guess after calling cyborg/neutron, nova will have filter hosts by names to get allocation candidates?
10:24:25	Kevin_Zheng	if we have specific network/accelerator requests?
10:24:35	tonyb	mriedem: okay I'll look over them
10:28:37	exarr	Anyone able to tell me, if I try to create a server, and it fails for $random_reason and hits an error state - is there a good way to retry the rebuild?
10:28:48	gibi	Kevin_Zheng: whta do you mean by specific accelerator request?
10:28:49	exarr	(Currently I delete, and supply all the detail over again)
10:28:52	openstackgerrit	Jianghua Wang proposed openstack/nova master: XenAPI: provide VGPU_DISPLAY_HEAD inventory in compute node https://review.openstack.org/523342
10:29:47	Kevin_Zheng	gibi create instances with accelerator=xxx in the flavor extra_spec
10:29:51	Kevin_Zheng	for example
10:31:15	gibi	Kevin_Zheng: is that xxx a type of accelerator or one given physical accelerator instance on some specific compute?
10:31:42	Kevin_Zheng	type
10:32:31	Kevin_Zheng	nova and cyborg will try to allocate the instance to the most suitable host
10:32:41	Kevin_Zheng	in whatever way
10:35:12	gibi	Kevin_Zheng: then in your example it is a VM boot case. In that case nova has to get the needed resources/traits for the accelerator either from cyborg or alternatively from flavor extra spec and include that in the GET allocation_candidate request for placement
10:35:49	gibi	Kevin_Zheng: then placement will return candidates that satisfy both nova and cyborg related resouces
10:38:02	Kevin_Zheng	gibi, yeah, as for now cyborg can only connect their own resource providers with nova resource providers by name, maybe the filter by hostname should be added to nova allocation candidates process together with CPU/RAM/Disk filtering?
10:41:55	gibi	Kevin_Zheng: cyborg connect the accelerator RPs to the related compute RPs by name, then nova asks placement, where are those RPs (RP trees) that satisfy the resource request (that includes both cpu/ram/disk and accelerator resources/traits) then so placement will already only return those RP trees that can provide both cpu/ram/disk and accelerator resources. No additional filtering is needed by nova
10:42:11	gmann_	gibi: we are discussing job template and irrelevant files in Davin suit 4th floor, if you want to join
10:42:37	gibi	gmann_: thanks for the heads up. on my way
10:42:58	gmann_	cool
10:43:24	Kevin_Zheng	gibi got it
10:49:10	openstackgerrit	Chris Dent proposed openstack/nova-specs master: Provide error codes for placement API https://review.openstack.org/418393
10:53:05	stelucz_	Hello is there any way how to update or sync `OS-EXT-SRV-ATTR:host` and `OS-EXT-SRV-ATTR:hypervisor_hostname` properties? because `nova show` reports different hypervisor than where vm is hosted. This happened after live migration. Thanks
11:04:20	Kevin_Zheng	Just poping to my head, since we have instance task states like networking, block_device_mapping, we might also have something for FPGA?
11:07:00	mriedem	fpga-a-ing
11:07:08	mriedem	accelerating
11:08:10	melwitt	I vote for fpga-a-ing
11:08:47	gibi	Kevin_Zheng: I think we have networking as we instruct neutron to bind ports. so the question for FPGA case is do nova needs to "bind" FPGAs?
11:10:24	Kevin_Zheng	gibi I thought the idea is to call cyborg to do "bind" actions in nova-compute? but correct me if I'm wrong
11:11:19	gibi	Kevin_Zheng: I'm not up to date about that part of the cyborg-nova interaction
11:12:10	Kevin_Zheng	mridem decelerating for detach?
11:12:20	Kevin_Zheng	:)
11:15:53	Kevin_Zheng	gibi, I think we can find it out this afternoon
11:19:49	openstackgerrit	Takashi NATSUME proposed openstack/nova master: Remove unused LOG variables https://review.openstack.org/546097

Earlier   Later