| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-27 | |||
| 09:53:23 | gibi | alex_xu: I think about it like adding volumes to an existing VMs | |
| 09:53:43 | efried | ^ | |
| 09:53:52 | efried | or attach_interface() | |
| 09:54:03 | alex_xu | efried: gibi, got it | |
| 09:54:05 | efried | 'cept not done by nova :) So gibi's example is much better :) | |
| 09:54:32 | alex_xu | so the cyborg calling the allocation candidates API to get a accelerator from the pool of the host? | |
| 09:54:39 | gibi | efried: thanks, I think I got it. post-deploy sounds like an easy shortcut for not getting tangled into the nova boot process | |
| 09:54:46 | alex_xu | since the instance already boot up on specific host | |
| 09:54:48 | efried | gibi: ++ | |
| 09:55:26 | efried | alex_xu: Yes. TBH, I'm not completely sure how one would call GET /allocation_candidates in such a way that it restricts to a certain host. | |
| 09:55:57 | alex_xu | efried: the we have parameter like in_tree? | |
| 09:55:59 | gibi | efried: it might be part of the 'anchor a request' discussion | |
| 09:56:04 | efried | alex_xu: Just so. | |
| 09:56:36 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Stop using mox in virt/xenapi/image/test_vdi_through_dev.py https://review.openstack.org/342682 | |
| 09:56:37 | efried | alex_xu: It depends to some extent on how the accelerators are modeled. If they're included in the compute node's provider tree, then yes, ?in_tree=<compute_uuid> on GET /allocation_candidates would be very appropriate. | |
| 09:57:18 | alex_xu | efried: yea | |
| 09:57:33 | efried | alex_xu: And/or if the accelerator is made part of an aggregate (not sure if that makes sense ever, but it extends to the more general model for e.g. bandwidth providers) then member_of | |
| 09:57:47 | efried | The latter is a Rocky bp. The former is not (yet). | |
| 09:58:42 | Kevin_Zheng | Any idea how Nova can consume resource reported by cyborg? | |
| 09:59:20 | alex_xu | efried: I didn't get the accelerator is made part of an aggregate. The aggregate in Nova? | |
| 09:59:51 | openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Move resource provider objects into placement hierarchy https://review.openstack.org/540049 | |
| 09:59:52 | openstackgerrit | Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435 | |
| 09:59:52 | openstackgerrit | Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766 | |
| 10:00:05 | alex_xu | Kevin_Zheng: depends on the direction we go I guess, with bauzas way, it can be done by cyborg API to return some resource class and traits. With the post-deploy attach, that is cyborg own thing | |
| 10:00:10 | bauzas | folks, I'm in the Scientifig WG room so I can't follow the convo | |
| 10:00:21 | efried | alex_xu: It probably doesn't make sense for accelerators to be modeled as true sharing providers. | |
| 10:00:23 | alex_xu | bauzas: no worries | |
| 10:00:43 | efried | alex_xu: Though be aware that sharing providers aren't the only possible use case for aggregates. | |
| 10:01:20 | efried | alex_xu: In the case of accelerators, which are pieces of hardware physically attached to the host, it most likely makes sense for them to be somewhere in the compute node's provider tree. | |
| 10:02:25 | efried | alex_xu: But for something like a bandwidth provider, where the component doing the work is maybe an external switch that manages traffic from multiple hosts, a sharing provider might be totally appropriate. (I need to reread the network bandwidth provider spec.) | |
| 10:02:26 | alex_xu | efried: yea, I can't image how accelerators shared between hosts | |
| 10:02:35 | gibi | Kevin_Zheng: as a wild alternative there is flavor extra_spec https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/custom-resource-classes-in-flavors.html | |
| 10:03:08 | alex_xu | efried: ah, that sounds make sense for bindwidth | |
| 10:03:10 | gibi | efried: correct, a TOR switch would be a shared provider | |
| 10:03:11 | efried | gibi: Here's a not-so-secret secret - that's how PowerVM is planning to do accelerator attach. | |
| 10:03:24 | efried | at least tentatively | |
| 10:03:35 | bauzas | efried: how the FPGA accelerators are modeled looks to me a Cyborg-only discusion | |
| 10:03:51 | efried | gibi: Though we may or may not need custom RCs. | |
| 10:03:54 | gibi | efried: We had our internal plans to do bandwidth with flavor extra_spec but it got voted down internally | |
| 10:04:06 | bauzas | I mean, I like the idea that the folks model their resources like they want and just allocate the resources like they want too | |
| 10:04:19 | efried | bauzas: ++ | |
| 10:04:23 | Kevin_Zheng | gibi, alex_xu so cyborg is in charge of reporting accelerator resources, but how should it be connected to Nova Hosts? | |
| 10:04:43 | efried | Kevin_Zheng: That's a great question. | |
| 10:04:50 | bauzas | the only Nova piece would be the fact to tie the instance to the compute that has the accelerated resource and do the binding | |
| 10:05:03 | gibi | bauzas: the allocation part getting tricky if we want to avoid races during boot. With post-attach it seems a lot easyer | |
| 10:05:11 | melwitt | fyi we have some time scheduled to chat with the cyborg team at 2pm this afternoon at the cyborg room | |
| 10:05:11 | efried | bauzas, Kevin_Zheng: Like attach_interface, but for device? | |
| 10:05:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Stop using mox in api/openstack/fakes.py https://review.openstack.org/325759 | |
| 10:05:38 | gibi | melwitt: yeah, I'm warming up for that now :) | |
| 10:05:44 | bauzas | melwitt: gtk, I wasn't knowing that | |
| 10:05:56 | bauzas | melwitt: I just spent 2 hours of my time already in that room :p | |
| 10:07:07 | gibi | Kevin_Zheng: nova creates the compute RP and names it after the host name of the compute so if cyborg knows the host name where the accelerator is then cyborg can look up the compute RP created by nova in placement by the host name | |
| 10:07:33 | alex_xu | gibi: ++ | |
| 10:07:36 | melwitt | oh, okay. on the dev ML last week or the week before we had a thread where we agreed on some time to discuss nova interaction things with the cyborg team | |
| 10:08:18 | melwitt | apologies if I didn't communicate that well enough, I put it on our ptg etherpad agenda times last week when I added times | |
| 10:08:29 | Kevin_Zheng | gibi hmm | |
| 10:08:36 | bauzas | melwitt: my bad, I probably messed up reading that ;) | |
| 10:08:53 | bauzas | melwitt: 2pm in their room ? | |
| 10:09:17 | melwitt | yes | |
| 10:10:05 | gibi | Kevin_Zheng: at least the host name will be the key information when neturon will create the networking RPs under the compute RP | |
| 10:10:20 | gibi | Kevin_Zheng: so I think the same can be done in cyborg | |
| 10:10:42 | mriedem | nova passes neutron the host name in the port binding so neutron can lookup the compute node RP yeah | |
| 10:10:43 | Kevin_Zheng | gibi yeah, I can see the picture now | |
| 10:10:46 | mriedem | for ironic, not so much... | |
| 10:12:13 | melwitt | FYI on thursday we're gonna start nova/cinder at 9:15 instead of 9:00 | |
| 10:12:33 | alex_xu | gibi: I guess in the future, the accelerator device also has the NUMA affinity thing, so the cyborg also need to look up the NUMA rp | |
| 10:12:53 | alex_xu | at that time, NUMA rp should have some naming rule also, i guess | |
| 10:14:02 | mriedem | need another core on this https://review.openstack.org/#/c/526772/ - leaking passwords isn't good | |
| 10:15:34 | gibi | mriedem: the network RP during the init of the neturon agent running on the given compute host, so that agent knows the host name without any nova involvement | |
| 10:16:01 | gibi | mriedem: the network RP is created during... | |
| 10:16:01 | mriedem | gibi: true | |
| 10:16:17 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] use simple FaultWrapper https://review.openstack.org/533752 | |
| 10:16:18 | openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Refactor WSGI apps and utils to limit imports https://review.openstack.org/533797 | |
| 10:16:47 | mriedem | lyarwood: tonyb: we have some stable/ocata changes that need another +2 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/ocata+label:Code-Review=2 | |
| 10:17:42 | gibi | mriedem: but yeah, if we talk about letting neutron create the bandwidth allocation in placement then the host name in the binding helps. However I'm more favor of duing the allocation from nova according to the selected allocation candidate | |
| 10:18:34 | gibi | alex_xu: yeah, NUMA is a good question for me. We have to make NUMA RP named after some host level id that can be known by other agents | |
| 10:21:48 | gibi | 10:58 < frickler> infra-root: seeing bumpy response times from gerrit, increasing GC times, too. maybe | |
| 10:21:51 | gibi | restart during PTG lunch time? | |
| 10:22:07 | gibi | alex_xu: so gerrit is slow for everybody | |
| 10:22:46 | jhesketh | gibi: it's being restarted now | |
| 10:24:03 | Kevin_Zheng | So I guess after calling cyborg/neutron, nova will have filter hosts by names to get allocation candidates? | |
| 10:24:25 | Kevin_Zheng | if we have specific network/accelerator requests? | |
| 10:24:35 | tonyb | mriedem: okay I'll look over them | |
| 10:28:37 | exarr | Anyone able to tell me, if I try to create a server, and it fails for $random_reason and hits an error state - is there a good way to retry the rebuild? | |
| 10:28:48 | gibi | Kevin_Zheng: whta do you mean by specific accelerator request? | |
| 10:28:49 | exarr | (Currently I delete, and supply all the detail over again) | |
| 10:28:52 | openstackgerrit | Jianghua Wang proposed openstack/nova master: XenAPI: provide VGPU_DISPLAY_HEAD inventory in compute node https://review.openstack.org/523342 | |
| 10:29:47 | Kevin_Zheng | gibi create instances with accelerator=xxx in the flavor extra_spec | |
| 10:29:51 | Kevin_Zheng | for example | |
| 10:31:15 | gibi | Kevin_Zheng: is that xxx a type of accelerator or one given physical accelerator instance on some specific compute? | |
| 10:31:42 | Kevin_Zheng | type | |
| 10:32:31 | Kevin_Zheng | nova and cyborg will try to allocate the instance to the most suitable host | |
| 10:32:41 | Kevin_Zheng | in whatever way | |
| 10:35:12 | gibi | Kevin_Zheng: then in your example it is a VM boot case. In that case nova has to get the needed resources/traits for the accelerator either from cyborg or alternatively from flavor extra spec and include that in the GET allocation_candidate request for placement | |
| 10:35:49 | gibi | Kevin_Zheng: then placement will return candidates that satisfy both nova and cyborg related resouces | |
| 10:38:02 | Kevin_Zheng | gibi, yeah, as for now cyborg can only connect their own resource providers with nova resource providers by name, maybe the filter by hostname should be added to nova allocation candidates process together with CPU/RAM/Disk filtering? | |
| 10:41:55 | gibi | Kevin_Zheng: cyborg connect the accelerator RPs to the related compute RPs by name, then nova asks placement, where are those RPs (RP trees) that satisfy the resource request (that includes both cpu/ram/disk and accelerator resources/traits) then so placement will already only return those RP trees that can provide both cpu/ram/disk and accelerator resources. No additional filtering is needed by nova | |
| 10:42:11 | gmann_ | gibi: we are discussing job template and irrelevant files in Davin suit 4th floor, if you want to join | |
| 10:42:37 | gibi | gmann_: thanks for the heads up. on my way | |
| 10:42:58 | gmann_ | cool | |