| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-27 | |||
| 15:18:31 | bauzas | edleafe: if each inventory is behind a child RP | |
| 15:18:38 | bauzas | either way, sounds we're in agreement | |
| 15:18:42 | edleafe | yeah | |
| 15:18:51 | bauzas | I *don't* want to make things complicated | |
| 15:19:03 | bauzas | the problem is more about the specific config option | |
| 15:19:08 | bauzas | I understand jaypipes on that | |
| 15:19:15 | jaypipes | bauzas: it would be the same inventory of VGPU. But different child providers would be tagged with specific VGPU_TYPE_XXX traits, right? | |
| 15:19:37 | sean-k-mooney[m] | stephenfin: if they are not associate with a guest im not sure. when they do get added to a guest they might also change. | |
| 15:19:41 | bauzas | jaypipes: you're talking of the possibility to have children, each of them being a type ? | |
| 15:19:56 | bauzas | jaypipes: if so, each inventory would be different | |
| 15:20:06 | jaypipes | bauzas: no | |
| 15:20:10 | bauzas | because the total number of vGPUs you can create depends on your type | |
| 15:20:20 | jaypipes | bauzas: I'm saying the resource class would all be "VGPU" | |
| 15:20:21 | bauzas | oh, with the conf opt solution ? | |
| 15:20:34 | stephenfin | sean-k-mooney[m]: Yeah, it seems like a lot of magic (even for ovs-dpdk) to say "we have this route that uses this NIC, therefore the vhost-user port should be processed by this PMD thread" | |
| 15:20:39 | bauzas | yeah, it's still VGPU resource class and a trait for that type | |
| 15:20:49 | jaypipes | bauzas: right. | |
| 15:20:58 | bauzas | so each PGPU will be a leaf | |
| 15:21:11 | sean-k-mooney[m] | stephenfin: this wont work in general however as this would only work with vhost-user. the approch you need to enable need to work for any switch backend | |
| 15:21:22 | bauzas | with one inventory about the total number of vGPUs it can create *for the type defined by the opt* | |
| 15:21:25 | jaypipes | bauzas: each pGPU group, but yes. | |
| 15:21:33 | bauzas | plus the trait telling which type it is | |
| 15:21:42 | bauzas | jaypipes: libvirt doesn't have the notion of groups | |
| 15:21:43 | jaypipes | bauzas: right. | |
| 15:21:49 | jaypipes | bauzas: I know, but xen does. | |
| 15:22:21 | bauzas | sure, but from a placement perspective, a "PGPU" RP is, from a libvirt perspective, a PCI device, and from a xen perspective, a PGPU group | |
| 15:22:25 | stephenfin | sean-k-mooney[m]: Sure, but vhost-user would be a start. Once we have a way to expose this information, we can extend other backends | |
| 15:22:30 | bauzas | but both are reconciled | |
| 15:22:37 | jaypipes | ack | |
| 15:22:41 | bauzas | it's just a driver-only thing | |
| 15:23:00 | bauzas | okay, now question | |
| 15:23:07 | bauzas | does that need to be spec'd up ? | |
| 15:23:11 | bauzas | jaypipes:^ | |
| 15:23:57 | sean-k-mooney[m] | stephenfin: if we wanted to do anything regardign the interface rx queue it likely should be an os-vif thing where we calulate teh best pmd our selves and set that not the other way around | |
| 15:24:13 | jaypipes | bauzas: yes. new CONF option, new way of behaving for the virt drivers. I would say yes. | |
| 15:24:35 | openstackgerrit | Merged openstack/nova master: tox: Remove unnecessary configuration https://review.openstack.org/556544 | |
| 15:26:09 | sean-k-mooney[m] | stephenfin: most backend wont that this info so vhost-user is not something we should build on | |
| 15:26:36 | stephenfin | sean-k-mooney[m]: What other backends would there be? | |
| 15:26:54 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Allow to specify granular CPU feature flags https://review.openstack.org/534384 | |
| 15:26:57 | sean-k-mooney[m] | stephenfin: for example kernel ovs with kernel vhost has kerne vhost-treads it spwans per interface, it dose not have pmds, so while we can taskset those threads we cant affinites them via looking a queues | |
| 15:27:18 | bauzas | jaypipes: fine by me | |
| 15:27:51 | bauzas | artom: had some concerns about the virt driver "calling" the conductor in https://review.openstack.org/#/c/552722/6 | |
| 15:28:24 | artom | bauzas, replied on the patch, but we can continue here if you want. Basically, no new calls are added, it's just existing methods/returns with new data in them | |
| 15:28:30 | bauzas | artom: maybe I misunderstood but you mean that the compute service would call back the conductor ? | |
| 15:28:30 | artom | Should I clarify the spec? | |
| 15:28:31 | alex_xu_ | kashyap: np | |
| 15:28:45 | bauzas | artom: that would help | |
| 15:29:03 | edmondsw | mriedem I don't remember ever discussing a service-protected-server case in our RBAC discussions, but I don't think you need to use service tokens there... just have a role that's given permission to manage these | |
| 15:29:12 | sean-k-mooney[m] | stephenfin: in any case queue mapping or tasksetting of vhost threads i think is out of scope of nova | |
| 15:29:26 | edmondsw | that spec needs a lot of work if anything's to happen there | |
| 15:30:10 | bauzas | artom: also, on the fact we would check the compute version | |
| 15:30:32 | bauzas | artom: does a live migration work with two different compute versions already ? | |
| 15:30:34 | openstackgerrit | Merged openstack/nova master: Move placement test cases from db to placement https://review.openstack.org/553149 | |
| 15:30:40 | mriedem | bauzas: yes | |
| 15:30:48 | mriedem | we support mixed version computes for live migration | |
| 15:30:49 | artom | bauzas, I didn't see anything in the code that would suggest it wouldn't | |
| 15:30:58 | bauzas | mriedem: context is https://review.openstack.org/#/c/552722/6/specs/rocky/approved/numa-aware-live-migration.rst@244 | |
| 15:30:59 | sean-k-mooney[m] | linux bridge, sriov, hardware offloaded ovs, vpp, iovision(ebpf), mini net, calico and macvtap are the main ones used beyond ovs/ovs-dpdk | |
| 15:30:59 | mriedem | we have a grenade + live migratoin job that also does this back and forth | |
| 15:31:25 | bauzas | mriedem: then, I was claiming we could microversion the behavioural change artom is going to introduce | |
| 15:31:26 | sean-k-mooney[m] | oh there is also snabb switch | |
| 15:31:27 | mriedem | that's a bit drastic | |
| 15:31:32 | artom | bauzas, also, we'd only fail it if the instance has "NUMA" characteristics | |
| 15:31:36 | mriedem | sec | |
| 15:31:50 | artom | Which is currently broken anyways, so not as drastic as it sounds | |
| 15:31:55 | mriedem | see https://review.openstack.org/#/c/522537/13/nova/conductor/tasks/live_migrate.py | |
| 15:31:55 | bauzas | artom: I agree, and we don't test that AFAIK | |
| 15:32:06 | mriedem | you only need to know if the source and dest computes can do the new hotness | |
| 15:32:35 | mriedem | otherwise it's a novalidhost if you can't find a pair | |
| 15:32:42 | bauzas | mriedem: yeah I remember that change | |
| 15:33:29 | sean-k-mooney[m] | stephenfin: on and NTT have a dpdk soft patch panel thing , the point is nova cant have special case code for all of these | |
| 15:35:21 | bauzas | mriedem: if the implementation of artom's spec would go into that direction, then yes we wouldn't need a microversion | |
| 15:35:35 | mriedem | i think we can agree that we don't want to add subnet to the requested network turducken in the compute api https://review.openstack.org/#/c/518227/ | |
| 15:37:11 | bauzas | mriedem: oh please, yes | |
| 15:37:33 | bauzas | just create the port in neutron and pass it to nova | |
| 15:37:50 | bauzas | aha, jinxed by mriedem's comment | |
| 15:37:58 | artom | mriedem, that link you posted in my NUMA migration spec, that's basically just checking min compute version, right? | |
| 15:38:04 | artom | So the same thing I was suggesting? | |
| 15:38:35 | artom | Just for clarity in my head, because I feel like we're talking about the same thing as though they were different | |
| 15:39:00 | gibi | jaypipes: I'm open to debate to simulate or not the neutron backend ordering preference via a scheduler weigher but for that debate we need some neutron heavy guys, like mlavalle | |
| 15:39:45 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Handle nested providers for allocation candidates https://review.openstack.org/556873 | |
| 15:39:47 | bauzas | artom: yeah basically | |
| 15:39:55 | bauzas | artom: it's for a different usage tho | |
| 15:39:56 | jaypipes | mriedem, bauzas: ^^ added not about microversion. | |
| 15:40:05 | artom | bauzas, right, but the mechanism is the same | |
| 15:40:08 | stephenfin | sean-k-mooney[m]: Well damn. That's unfortunate | |
| 15:40:08 | kashyap | Darn, lost edits to wiki.openstack.org, as it logged me out mid-way. And didn't give me a way to get the changes back. | |
| 15:40:27 | bauzas | artom: yup, the pattern should be the same | |
| 15:40:29 | artom | (We might eventually standardise that in a utils somewhere, since we seem to be doing a lot of it, btw) | |
| 15:40:39 | mriedem | artom: kind of, but it depends on where you're using it, | |
| 15:41:01 | mriedem | if it's the API and you're checking min compute service version across the entire cell, that's different than just comparing the source and candidate dest host | |
| 15:41:16 | bauzas | at least, we can assume we run a pre-flight check on the conductor that checks both compute versions | |
| 15:41:27 | bauzas | that could be generalized | |
| 15:41:38 | artom | mriedem, ah, so you're saying we should be more granular and just check the (source, dest) pair | |
| 15:41:41 | artom | Hrmm | |
| 15:41:46 | artom | Could we do that in the scheduler? | |
| 15:41:48 | mriedem | artom: yes, see that patch i linked | |
| 15:41:51 | bauzas | but that's an implementation detail, IHMO | |
| 15:41:58 | artom | So that straight away we have a dest that supports it? | |
| 15:42:07 | mriedem | artom: we could if we exposed the capability as a trait on the compute and added a pre-request filter | |