Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-27
10:00:10 bauzas folks, I'm in the Scientifig WG room so I can't follow the convo
10:00:21 efried alex_xu: It probably doesn't make sense for accelerators to be modeled as true sharing providers.
10:00:23 alex_xu bauzas: no worries
10:00:43 efried alex_xu: Though be aware that sharing providers aren't the only possible use case for aggregates.
10:01:20 efried alex_xu: In the case of accelerators, which are pieces of hardware physically attached to the host, it most likely makes sense for them to be somewhere in the compute node's provider tree.
10:02:25 efried alex_xu: But for something like a bandwidth provider, where the component doing the work is maybe an external switch that manages traffic from multiple hosts, a sharing provider might be totally appropriate. (I need to reread the network bandwidth provider spec.)
10:02:26 alex_xu efried: yea, I can't image how accelerators shared between hosts
10:02:35 gibi Kevin_Zheng: as a wild alternative there is flavor extra_spec https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/custom-resource-classes-in-flavors.html
10:03:08 alex_xu efried: ah, that sounds make sense for bindwidth
10:03:10 gibi efried: correct, a TOR switch would be a shared provider
10:03:11 efried gibi: Here's a not-so-secret secret - that's how PowerVM is planning to do accelerator attach.
10:03:24 efried at least tentatively
10:03:35 bauzas efried: how the FPGA accelerators are modeled looks to me a Cyborg-only discusion
10:03:51 efried gibi: Though we may or may not need custom RCs.
10:03:54 gibi efried: We had our internal plans to do bandwidth with flavor extra_spec but it got voted down internally
10:04:06 bauzas I mean, I like the idea that the folks model their resources like they want and just allocate the resources like they want too
10:04:19 efried bauzas: ++
10:04:23 Kevin_Zheng gibi, alex_xu so cyborg is in charge of reporting accelerator resources, but how should it be connected to Nova Hosts?
10:04:43 efried Kevin_Zheng: That's a great question.
10:04:50 bauzas the only Nova piece would be the fact to tie the instance to the compute that has the accelerated resource and do the binding
10:05:03 gibi bauzas: the allocation part getting tricky if we want to avoid races during boot. With post-attach it seems a lot easyer
10:05:11 melwitt fyi we have some time scheduled to chat with the cyborg team at 2pm this afternoon at the cyborg room
10:05:11 efried bauzas, Kevin_Zheng: Like attach_interface, but for device?
10:05:19 openstackgerrit Stephen Finucane proposed openstack/nova master: Stop using mox in api/openstack/fakes.py https://review.openstack.org/325759
10:05:38 gibi melwitt: yeah, I'm warming up for that now :)
10:05:44 bauzas melwitt: gtk, I wasn't knowing that
10:05:56 bauzas melwitt: I just spent 2 hours of my time already in that room :p
10:07:07 gibi Kevin_Zheng: nova creates the compute RP and names it after the host name of the compute so if cyborg knows the host name where the accelerator is then cyborg can look up the compute RP created by nova in placement by the host name
10:07:33 alex_xu gibi: ++
10:07:36 melwitt oh, okay. on the dev ML last week or the week before we had a thread where we agreed on some time to discuss nova interaction things with the cyborg team
10:08:18 melwitt apologies if I didn't communicate that well enough, I put it on our ptg etherpad agenda times last week when I added times
10:08:29 Kevin_Zheng gibi hmm
10:08:36 bauzas melwitt: my bad, I probably messed up reading that ;)
10:08:53 bauzas melwitt: 2pm in their room ?
10:09:17 melwitt yes
10:10:05 gibi Kevin_Zheng: at least the host name will be the key information when neturon will create the networking RPs under the compute RP
10:10:20 gibi Kevin_Zheng: so I think the same can be done in cyborg
10:10:42 mriedem nova passes neutron the host name in the port binding so neutron can lookup the compute node RP yeah
10:10:43 Kevin_Zheng gibi yeah, I can see the picture now
10:10:46 mriedem for ironic, not so much...
10:12:13 melwitt FYI on thursday we're gonna start nova/cinder at 9:15 instead of 9:00
10:12:33 alex_xu gibi: I guess in the future, the accelerator device also has the NUMA affinity thing, so the cyborg also need to look up the NUMA rp
10:12:53 alex_xu at that time, NUMA rp should have some naming rule also, i guess
10:14:02 mriedem need another core on this https://review.openstack.org/#/c/526772/ - leaking passwords isn't good
10:15:34 gibi mriedem: the network RP during the init of the neturon agent running on the given compute host, so that agent knows the host name without any nova involvement
10:16:01 gibi mriedem: the network RP is created during...
10:16:01 mriedem gibi: true
10:16:17 openstackgerrit Chris Dent proposed openstack/nova master: [placement] use simple FaultWrapper https://review.openstack.org/533752
10:16:18 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Refactor WSGI apps and utils to limit imports https://review.openstack.org/533797
10:16:47 mriedem lyarwood: tonyb: we have some stable/ocata changes that need another +2 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/ocata+label:Code-Review=2
10:17:42 gibi mriedem: but yeah, if we talk about letting neutron create the bandwidth allocation in placement then the host name in the binding helps. However I'm more favor of duing the allocation from nova according to the selected allocation candidate
10:18:34 gibi alex_xu: yeah, NUMA is a good question for me. We have to make NUMA RP named after some host level id that can be known by other agents
10:21:48 gibi 10:58 < frickler> infra-root: seeing bumpy response times from gerrit, increasing GC times, too. maybe
10:21:51 gibi restart during PTG lunch time?
10:22:07 gibi alex_xu: so gerrit is slow for everybody
10:22:46 jhesketh gibi: it's being restarted now
10:24:03 Kevin_Zheng So I guess after calling cyborg/neutron, nova will have filter hosts by names to get allocation candidates?
10:24:25 Kevin_Zheng if we have specific network/accelerator requests?
10:24:35 tonyb mriedem: okay I'll look over them
10:28:37 exarr Anyone able to tell me, if I try to create a server, and it fails for $random_reason and hits an error state - is there a good way to retry the rebuild?
10:28:48 gibi Kevin_Zheng: whta do you mean by specific accelerator request?
10:28:49 exarr (Currently I delete, and supply all the detail over again)
10:28:52 openstackgerrit Jianghua Wang proposed openstack/nova master: XenAPI: provide VGPU_DISPLAY_HEAD inventory in compute node https://review.openstack.org/523342
10:29:47 Kevin_Zheng gibi create instances with accelerator=xxx in the flavor extra_spec
10:29:51 Kevin_Zheng for example
10:31:15 gibi Kevin_Zheng: is that xxx a type of accelerator or one given physical accelerator instance on some specific compute?
10:31:42 Kevin_Zheng type
10:32:31 Kevin_Zheng nova and cyborg will try to allocate the instance to the most suitable host
10:32:41 Kevin_Zheng in whatever way
10:35:12 gibi Kevin_Zheng: then in your example it is a VM boot case. In that case nova has to get the needed resources/traits for the accelerator either from cyborg or alternatively from flavor extra spec and include that in the GET allocation_candidate request for placement
10:35:49 gibi Kevin_Zheng: then placement will return candidates that satisfy both nova and cyborg related resouces
10:38:02 Kevin_Zheng gibi, yeah, as for now cyborg can only connect their own resource providers with nova resource providers by name, maybe the filter by hostname should be added to nova allocation candidates process together with CPU/RAM/Disk filtering?
10:41:55 gibi Kevin_Zheng: cyborg connect the accelerator RPs to the related compute RPs by name, then nova asks placement, where are those RPs (RP trees) that satisfy the resource request (that includes both cpu/ram/disk and accelerator resources/traits) then so placement will already only return those RP trees that can provide both cpu/ram/disk and accelerator resources. No additional filtering is needed by nova
10:42:11 gmann_ gibi: we are discussing job template and irrelevant files in Davin suit 4th floor, if you want to join
10:42:37 gibi gmann_: thanks for the heads up. on my way
10:42:58 gmann_ cool
10:43:24 Kevin_Zheng gibi got it
10:49:10 openstackgerrit Chris Dent proposed openstack/nova-specs master: Provide error codes for placement API https://review.openstack.org/418393
10:53:05 stelucz_ Hello is there any way how to update or sync `OS-EXT-SRV-ATTR:host` and `OS-EXT-SRV-ATTR:hypervisor_hostname` properties? because `nova show` reports different hypervisor than where vm is hosted. This happened after live migration. Thanks
11:04:20 Kevin_Zheng Just poping to my head, since we have instance task states like networking, block_device_mapping, we might also have something for FPGA?
11:07:00 mriedem fpga-a-ing
11:07:08 mriedem accelerating
11:08:10 melwitt I vote for fpga-a-ing
11:08:47 gibi Kevin_Zheng: I think we have networking as we instruct neutron to bind ports. so the question for FPGA case is do nova needs to "bind" FPGAs?
11:10:24 Kevin_Zheng gibi I thought the idea is to call cyborg to do "bind" actions in nova-compute? but correct me if I'm wrong
11:11:19 gibi Kevin_Zheng: I'm not up to date about that part of the cyborg-nova interaction
11:12:10 Kevin_Zheng mridem decelerating for detach?
11:12:20 Kevin_Zheng :)
11:15:53 Kevin_Zheng gibi, I think we can find it out this afternoon
11:19:49 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove unused LOG variables https://review.openstack.org/546097
11:32:30 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: mask InjectionInfo.admin_pass https://review.openstack.org/526772
11:36:35 bauzas Kevin_Zheng: gibi: mriedem: alex_xu: huh, just saw https://review.openstack.org/#/c/318047/6/specs/backlog/approved/fpga-use-cases.rst
11:36:58 openstackgerrit Murali Annamneni proposed openstack/nova master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446643
11:37:25 bauzas I refrain commenting here about late binding, but I just feel that's up to the compute manager to make the device available to the instance
11:37:54 bauzas the only thing we need to care is to make sure the host that the instance is going is having *that* accelerator
11:38:09 mriedem that is pretty old, and i think came up from another spec that was in cyborg that was abandoned
11:38:09 bauzas how the binding is done, meh to me
11:38:26 mriedem this https://review.openstack.org/#/c/448228/
11:38:33 bauzas it's a backlog spec, I'm fine with reviewing it
11:38:37 mriedem i guess that's actually newer

Earlier   Later