Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-05-30
14:55:21 stephenfin dansmith: Just to make sure, there isn't a deadline I need to wary of, in that case?
14:55:36 dansmith oh, maybe
14:55:58 stephenfin I'm also nervous of forgetting our learnings about not solving everything in the spec
14:56:17 stephenfin I think the code that I already have proves out the idea. Now I just need to shuffle about where I'm storing stuff
14:56:40 stephenfin (also, no spec = no runway slot)
15:01:00 bauzas gentle reminder for folks that the spec about multiple vGPU types has been updated https://review.openstack.org/#/c/557065/
15:01:06 bauzas jaypipes: dansmith: ^
15:02:21 mriedem stephenfin: spec freeze is june 7
15:02:25 mriedem so you've got about a week
15:02:33 stephenfin to...?
15:02:38 mriedem get the spec approved
15:03:13 stephenfin That's in your hands then, I guess? I'll work on updating the code in the interim
15:03:17 mriedem probably not a bad idea to do another spec review sprint/flush in the coming week to get anything we want into rocky through before spec freeze
15:03:34 mriedem my volume multiattach enhancement spec has been waiting too
15:04:48 bauzas mriedem: you can review this easy spec for just a conf opt https://review.openstack.org/#/c/557065/
15:05:11 mriedem i can add it to the queue
15:05:15 cfriesen mriedem: what are your thoughts on getting https://review.openstack.org/#/c/465787/ merged?
15:05:29 bauzas mriedem: fair enough
15:05:41 cfriesen mriedem: I guess you've got a todo in there still
15:05:52 openstackgerrit Vishakha Agarwal proposed openstack/python-novaclient master: No requirement of –all-tenants while listing servers https://review.openstack.org/569090
15:05:53 stephenfin mriedem: I don't know how much use I'd be reviewing that. I've only reviewed patches for that stuff because ildikov wouldn't stop bugging me :) I can try though
15:06:12 bauzas mriedem: stephenfin: FWIW, I'm tempted to punt https://review.openstack.org/#/c/552924/ from Rocky and defer to Stein
15:06:15 bauzas jaypipes: ^
15:06:43 bauzas because nested RPs is still in early bits, and because I shit have no time to further work on implementing it for Rocky anyway
15:06:57 bauzas so I'll work on providing a new rev' of course, but not prioritary
15:07:12 stephenfin bauzas: Wait, how does that compare with jaypipes' spec?
15:07:13 ildikov stephenfin: I can bug you again if that helps :)
15:07:14 jaypipes bauzas: I have a feeling tetsuro will have made significant progress on much of that in Rocky.
15:07:28 jaypipes stephenfin: the cpu-resources spec is about dedicated cores, not NUMA.
15:07:40 jaypipes dedicated and shared cores.
15:07:41 mriedem cfriesen: i would like to see it merged, so i don't have thoughts on my own patch :)
15:07:45 mriedem cfriesen: you should ask other cores
15:07:48 bauzas jaypipes: I looked at tetsuro's patches and those are still under review, right?
15:07:57 jaypipes bauzas: of course, yes
15:08:05 bauzas so, it's in my pipe
15:08:09 mriedem cfriesen: just throw it in the starlingx mega patch-o-doom and it will get reviews from at&t for sure!
15:08:11 stephenfin jaypipes: But you're using NUMA, no?
15:08:23 jaypipes bauzas: not just tetsuro, though. efried and Citrix guys are pushing ahead.
15:08:33 bauzas but my point is, nested resource providers is just incepted, and possibly vGPUs are the first usecases that'll use those
15:08:47 mriedem cfriesen: the todo is for future refactor
15:08:48 stephenfin jaypipes: e.g. https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@217
15:08:51 bauzas jaypipes: I'm working on that too, remember
15:08:52 jaypipes stephenfin: that latest revision I tried to remove as much ass possible any mentions of NUMA in so much as extra specs or anything about the NUMATopologyFilter.
15:08:59 mriedem cfriesen: if i meant to hold up the patch on the todo, i would have wip'ed it
15:09:07 cfriesen mriedem: the plan is to submit individual small patches... Actually, we're already using your patch so it's already in the list.
15:09:17 bauzas jaypipes: http://lists.openstack.org/pipermail/openstack-dev/2018-May/130783.html
15:09:18 mriedem cfriesen: heh i know
15:09:19 cfriesen mriedem: yeah, I caught that on re-read
15:09:29 bauzas jaypipes: you possibly missed it
15:09:30 jaypipes bauzas: I'm not holding up progress on n-r-p's various efforts ongoing.
15:09:33 mriedem cfriesen: i just need to stab the starlingx thing at every opportunity
15:09:35 jangutter sahid: Apologies for side-tracking, but I've got a question regarding the thorny issue of multiqueue and os-vif, if you have a few seconds.
15:09:43 cfriesen mriedem: bring it on, I can take it
15:09:45 jaypipes bauzas: I didn't miss it. just haven't had a chance to go through it.
15:09:51 jaypipes bauzas: will do today.
15:09:55 bauzas ack, thanks
15:09:58 jaypipes np
15:10:06 mriedem cfriesen: also, poutine, wtf?!
15:10:07 bauzas because I'm just litterally working on the implementation
15:10:20 cfriesen mriedem: good poutine is awesome. crappy poutine sucks
15:10:26 stephenfin jaypipes: I'm guessing you can't ignore NUMA completely though, if you want to support use case 4 and the likes? https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425
15:10:27 bauzas jaypipes: and I'm hold until we get consensus on the right approach
15:10:42 bauzas jaypipes: and FWIW, naichuan's implementation for Xen is impacted too
15:11:40 jaypipes stephenfin: no, I specifically *don't* want to include the realtime extra specs or any of the numa_nodes/numa_cells goo.
15:12:02 jaypipes stephenfin: because the purpose of that spec is to handle requests for dedicated and shared CPU resources. nothing to do with NUMA really.
15:12:17 sahid jangutter: i imagine what you are going to do it's add in nova a new condition for vif detail vrouter, which will enable the multiqueue=True
15:12:27 mriedem stephenfin: wasn't trying to bug you for reviews on my spec, just saying i'm in the same boat
15:12:30 sahid that under the section vhostuser
15:12:32 jaypipes stephenfin: and I'm deliberately trying to not bring in the awful coupling of NUMATopology and InstanceNUMATopology and the horrible extra specs that serve to construct that mess.
15:12:34 mriedem stephenfin: we can happily drown together
15:12:37 mriedem you can be leo
15:12:39 mriedem i'll be kate
15:12:49 mriedem / who didn't drown /
15:12:51 jaypipes mriedem: wow, how chivalrous of you :P
15:13:12 jangutter sahid: actually, the idea is to migrate the 'vrouter' vif which is not vhostuser...
15:13:14 openstackgerrit Merged openstack/nova-specs master: Libvirt file backed memory https://review.openstack.org/563704
15:13:37 stephenfin jaypipes: I've missed something so. I thought requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' means host NUMA topology would need to be accounted for
15:13:43 jangutter sahid: I did some checking, and it looks like multiqueue is an instance-wide setting, not a vif-specific one.
15:14:30 jaypipes stephenfin: no. it has nothing explicitly to do with NUMA. that just means "I need 2 groups of 8 dedicated CPU resources. they need to be provided by two distinct resource providers"
15:14:36 jangutter sahid: ref: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L708
15:15:21 jaypipes stephenfin: the fact that the compute node resource provider might be a parent to two other child providers that *represent* different sockets or NUMA cells is really not relevant to the request that two different providers of dedicated CPU supply the requested resource groups.
15:16:07 sahid jangutter: i'm not sure i understand what you want do?
15:16:20 jaypipes stephenfin: in other words, if some non-NUMA devices *happened* to expose PCPU inventory under a provider tree, they would meet the requirements...
15:16:25 bauzas jaypipes: stephenfin: just a point, we could have the same upgrade impact with VCPU resources being in child RPs than the one I described with http://lists.openstack.org/pipermail/openstack-dev/2018-May/130783.html
15:16:32 stephenfin jaypipes: Practically speaking so, requesting 'resources1:PCPU=8 resources2:PCPU=8 group_policy=isolate' will always fail until NUMA is modelled in placement?
15:16:40 stephenfin Ah, gotcha
15:16:41 jaypipes stephenfin: again, we are trying desperately to free ourselves of the NUMA coupling that has plagued all of this for years.
15:17:13 jangutter sahid: vrouter kernel mode plugging needs to create a tap with multiqueue enabled, if libvirt starts the VM up looking for it.
15:17:30 stephenfin jaypipes: But we're still coupled, no? I mean, we're going to need NUMA for the above request to work
15:17:42 bhagyashris efried: any suggestion about second approach
15:17:43 jaypipes stephenfin: I'm trying my best to treat NUMA as just the model of distances to various memory (and memory page) caches to a processor/bus, instead of coupling all that mess with CPU pinning.
15:18:07 jangutter sahid: the VIFGeneric portion can set the libvirt config correctly for multiqueue, but, it looks like it's going to be difficult to pass "multiqueue" on a per-vif basis without modding large portions of the os-vif plugging code.
15:18:09 sahid jangutter: ok so you plugin needs to know whether multiqueue is enable, which is via the os-vif object nova is filling then passing the the os-vif plugin
15:18:21 openstackgerrit Matt Riedemann proposed openstack/nova master: Add osprofiler config options to generated reference https://review.openstack.org/571225
15:18:51 jangutter sahid: yep, so, either we can pass "multiqueue=True" via the vif os-vif object, or via the instance os-vif object.
15:19:59 bauzas jaypipes: unfortunately from what I understood from operators, pinning is there because NUMA affinity misses for PCI devices
15:20:09 bauzas jaypipes: so both are somehow related
15:20:16 stephenfin jaypipes: To be clear though, use case 4 in that spec cannot be achieved with modelling NUMA in placement, right?
15:20:19 jangutter sahid: the trick is, passing it via the VIF looks very tricky. A lot of code has to be added there, since, paradoxically, the choice whether to enable multiqueue depends on the instance and flavor settings, not VIF settings.
15:20:37 stephenfin jaypipes: Use case 4 = https://review.openstack.org/#/c/555081/10/specs/rocky/approved/cpu-resources.rst@425

Earlier   Later