| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-27 | |||
| 14:42:40 | alex_xu_ | kashyap: actually I mean I didn't find a extra_flags field for the LibvirtConfigGuestCPU even for now... | |
| 14:42:55 | gibi | jaypipes: the backend traits are not needed for the placement query or for the scheduler filters | |
| 14:43:04 | efried | jaypipes: Do you have a spec queued up for NRP-in-a_c yet? | |
| 14:43:31 | gibi | jaypipes: but as the filter scheduler makes the allocation, it implicitly decides about the backend as well | |
| 14:43:49 | gibi | jaypipes: and before the bandwidth feature this decision was made by neutron | |
| 14:44:07 | gibi | jaypipes: but after it, the decision is made by the filter scheduler | |
| 14:44:11 | bhagyashris | mriedem, alex_xu_: thank you :) | |
| 14:44:23 | gibi | jaypipes: so we want to save some of the freedom of neutron here | |
| 14:44:25 | kashyap | alex_xu_: It's not in that class. But take a look at LibvirtConfigCPUFeature() | |
| 14:44:38 | gibi | jaypipes: by adding a weigher that can express backend preference order | |
| 14:45:28 | kashyap | alex_xu_: Typo, actually this one: LibvirtConfigGuestCPUFeature() | |
| 14:45:39 | jaypipes | efried: crap. haven't finished it. | |
| 14:45:44 | jaypipes | efried: I can push what I have. | |
| 14:46:08 | efried | jaypipes: as you see fit. Just thought I'd ask, it being spec review day and all. | |
| 14:47:06 | openstackgerrit | Matthew Booth proposed openstack/nova-specs master: Add serial numbers for local disks https://review.openstack.org/556565 | |
| 14:47:10 | jaypipes | efried: ack, thx for the reminder. | |
| 14:49:01 | sean-k-mooney[m] | stephenfin: hi sorry was in a meeting you wanted me to comment on some ovs-dpdk stuff | |
| 14:49:27 | openstackgerrit | Dan Smith proposed openstack/nova-specs master: Amend the member_of spec for multiple query sets https://review.openstack.org/555413 | |
| 14:49:58 | efried | dansmith: remove -2 from ^ ? | |
| 14:50:11 | dansmith | efried: yep, I just need to fix a pep8 thing I just noticed | |
| 14:50:16 | efried | o | |
| 14:50:27 | efried | dansmith: Two spots | |
| 14:51:10 | stephenfin | sean-k-mooney[m]: Indeed. sahid was suggesting we could simply rely on NUMA affinity of a vhost-user interface's PMD queues to determine where to place a host | |
| 14:51:12 | bauzas | mriedem: ack, thanks | |
| 14:51:23 | bauzas | mriedem: honestly, I wasn't knowing what to do with this | |
| 14:51:32 | efried | dansmith: While you're at it, "in in" | |
| 14:52:46 | dansmith | efried: ah yeah saw that before and had forgottten it | |
| 14:52:48 | dansmith | tttt | |
| 14:53:03 | efried | dansmith: Please clarify whether the multiple-member_of thing will also be implemented in GET /resource_providers | |
| 14:53:36 | dansmith | efried: ah, I was going to ask if we should do that too, but I expected it would be outside this spec, no? | |
| 14:54:32 | efried | dansmith: Whether it is or not, that should be stated. I could go either way. But slight preference for including it. For a couple of reasons... | |
| 14:54:39 | dansmith | ack | |
| 14:55:23 | efried | dansmith: First, a single microversion introducing multiple-member_of syntax. I like that better than one microversion introducing it for one URI, another for introducing what's effectively the same feature to another URI. | |
| 14:55:26 | openstackgerrit | Dan Smith proposed openstack/nova-specs master: Amend the member_of spec for multiple query sets https://review.openstack.org/555413 | |
| 14:55:54 | efried | dansmith: Second, it's actually going to make the code easier to write. Because we currently do the processing of member_of in common code for both; so we can continue to do that. | |
| 14:56:00 | dansmith | efried: sure | |
| 14:56:45 | sean-k-mooney[m] | stephenfin: that wont work als ovs has no idea if there is enough memory on the numa node for the vm also the memory is not allocated until the vhost-user frontend in qemu connects to the vhost-user backend in ovs | |
| 14:56:54 | efried | dansmith: Technically, the Work Items section should be updated... | |
| 14:57:15 | dansmith | efried: for /rps? | |
| 14:57:31 | efried | dansmith: On rereading, it's sufficiently vague to be acceptable as is. | |
| 14:57:36 | dansmith | yeah, I was going to say.. | |
| 14:57:45 | efried | dansmith: Which is fine by me; I've always thought that section was pretty much redundant anyway. | |
| 14:58:20 | efried | dansmith: +1. One spec down! | |
| 14:58:29 | sean-k-mooney[m] | stephenfin: the other thing is that when the vhost-user frontend connects to the vhost-user backend in ovs it tries to allocate memory and pmd for the vhost-user interface from on of the numa nodes of the guest automatically | |
| 14:58:43 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Handle nested providers for allocation candidates https://review.openstack.org/556873 | |
| 14:59:50 | jaypipes | efried: ^ | |
| 14:59:58 | efried | jaypipes: ack | |
| 15:00:53 | mriedem | bhagyashris: comments inline https://review.openstack.org/#/c/511825/ | |
| 15:04:00 | jaypipes | gibi: maybe I'm just being thick... I still don't get it. You will have multiple backends on the same compute host supporting the same physical networks that support the same vNIC types and you want to be able to choose which backend to allocate a piece of bandwidth from? | |
| 15:04:12 | bauzas | jaypipes: I'm not sure we need a spec for https://review.openstack.org/#/c/556873/1/specs/rocky/approved/nested-resource-providers-allocation-candidates.rst | |
| 15:04:24 | bauzas | jaypipes: it's just fixes we need to merge IMHO | |
| 15:05:01 | mriedem | bauzas: it's an api change and microversion right? | |
| 15:05:03 | mriedem | so spec is required yeah | |
| 15:05:05 | mriedem | ? | |
| 15:05:17 | gibi | jaypipes: exactly. Neutron today does it in the following way: | |
| 15:05:28 | gibi | jaypipes: Neutron has a mechnism driver config | |
| 15:05:32 | bauzas | mriedem: from the spec itself, looks like it's not changing the API | |
| 15:05:37 | stephenfin | sean-k-mooney[m]: Potentially dumb question, but in what way would lack of guest memory be related? | |
| 15:05:47 | gibi | jaypipes: Neutron iterates throught that list and try binding the port with the given driver | |
| 15:05:48 | jaypipes | mriedem: no API change, no. | |
| 15:06:05 | stephenfin | I figured if you could do 'ovs-appctl dpif-netdev/pmd-rxq-show' to show the affinity for a given interface and just return that | |
| 15:06:12 | gibi | jaypipes: the first driver that returns a positive result from that bind call will be the one Neutron use | |
| 15:06:30 | gibi | jaypipes: so that config option defines a preference order between backends supporting the same physnet | |
| 15:06:34 | mriedem | jaypipes: bauzas: it's a behavior change for the alloc candidates api though | |
| 15:06:35 | stephenfin | Or does that even exist before the port is attached to the interface? | |
| 15:06:45 | bhagyashris | mriedem: thank you for review i will look into it as i am working in IST time zone it's end of day for me :) | |
| 15:06:53 | stephenfin | this would be so much easier if I have a machine to experiment on. Stupid fried motherboard is killing me :( | |
| 15:07:07 | jaypipes | mriedem: if "behaviour change" means "it will work when there are nested providers", then yes. :) | |
| 15:07:12 | bauzas | mriedem: technically, alloc-candidates doesn't work yet with nested RPs | |
| 15:07:19 | jaypipes | stephenfin: at least it's not an efried motherboard. | |
| 15:07:25 | bauzas | mriedem: it's not changing the existing | |
| 15:07:35 | mriedem | bauzas: technically volume-backed rebuild with a new image doesn't work either, | |
| 15:07:40 | mriedem | but if we make that work, it's an api change | |
| 15:07:51 | mriedem | even if the request params don't change | |
| 15:08:01 | jaypipes | does it even matter guys? the spec is up. | |
| 15:08:01 | bauzas | I agree it's a signal | |
| 15:08:12 | mriedem | yeah i'm saying it's a spec, | |
| 15:08:15 | mriedem | and likely a microversion bump | |
| 15:08:26 | bauzas | okay, I'll comment that too then | |
| 15:09:47 | bauzas | mriedem: jaypipes: speaking of specs | |
| 15:10:03 | bauzas | jaypipes: now I'm back, can we discuss about my point with vGPU types ? | |
| 15:10:17 | bauzas | looks like you were sad about that | |
| 15:11:05 | sahid | stephenfin, sean-k-mooney[m] I think the issue is on the scheduling, we will have first to select a host so then we could create the ports and query them | |
| 15:11:28 | stephenfin | sahid: I'd be perfectly fine saying we need pre-created ports for this thing | |
| 15:11:54 | stephenfin | That's already a requirement for SR-IOV and afaik is the plan for gibi's bandwidth-aware scheduling spec | |
| 15:12:18 | sahid | stephenfin: the problem is then you need to raise that re-schedule excpetion is the resources are not enough | |
| 15:12:27 | jaypipes | bauzas: "sad" would be one way to put it, yes. | |
| 15:12:32 | sahid | yes i think we do that somewhere but i can't really remember | |
| 15:12:47 | stephenfin | same issue with SR-IOV though, right? | |
| 15:12:47 | bauzas | jaypipes: let's be gentlemen :p | |
| 15:13:08 | bauzas | jaypipes: so, before discussing about a solution, do you understand the problem ? | |
| 15:13:18 | sahid | stephenfin: yes probably you are right | |
| 15:13:43 | jaypipes | bauzas: yes, I fully understand the problem. | |
| 15:13:57 | bauzas | jaypipes: in https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html we don't mention that a single pGPU can have multiple types and we just suppose a single inventory of VGPU for each | |
| 15:14:47 | jaypipes | bauzas: I had numerous conversations with jianghuaw_ about this. | |
| 15:14:48 | bauzas | the fact is, once *one* mediated device is created, then *all* the others types are not possible for that pGPU | |
| 15:15:03 | stephenfin | sahid, sean-k-mooney[m]: The bigger concern I have is that a vhost-user port is configured to a given PMD based on the guest - not the routes | |
| 15:15:04 | bauzas | jaypipes: and what are your thoughts on that ? | |
| 15:15:52 | bauzas | jaypipes: we already somehow set inventories based on config option thru enabled_gpu_types tho | |
| 15:16:00 | jaypipes | bauzas: the only thing I really don't want is on-the-fly re-configuration of providers based on *what the user requested*. | |