Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-27
20:50:12 cfriesen sean-k-mooney[m]: can we map multiple files for an instance with a single virtual numa node?
20:50:53 sean-k-mooney[m] cfriesen: we just dont but we can numa afinites 4k pages
20:51:26 cfriesen sean-k-mooney[m]: yes, but can we allocate 3GB of 4K pages from host numa node 0 and 1GB from host numa node 1 for an instance with a single virtual numa node?
20:52:26 sean-k-mooney[m] cfriesen: i think so. why would you want too
20:52:34 sean-k-mooney[m] to allow the memory to come form multiple host numa nodes
20:53:13 cfriesen sean-k-mooney[m]: If I have only 3GB memory free on one numa node and 1GB on the other, and I want to keep the current behaviour of letting the instance float across the whole compute node.
20:54:09 jaypipes sean-k-mooney[m]: again, I think that granular request groups should mean that the resources in each request group are provided by different resource providers.
20:54:26 sean-k-mooney[m] i would have to check. we added the abiltiy to use file desciptor memory by setting the souce elemet of this https://libvirt.org/formatdomain.html#elementsMemoryBacking
20:54:29 jaypipes sean-k-mooney[m]: I do not care about the use case of "general spreadability".
20:54:32 efried jaypipes: but only if they're the same resource class
20:54:38 jaypipes efried: yes.
20:54:54 sean-k-mooney[m] i know that backing file can be affinites but i dont know if we can create two backing files attach to the same guest numa node
20:55:14 efried jaypipes: That's gonna be tough to implement, just for starters.
20:55:17 jaypipes efried: though I don't see a reason why you would separate request groups where one request group does *not* contain a resource class...
20:55:33 efried jaypipes: So that they aren't forced to land on the *same* RP.
20:55:34 sean-k-mooney[m] cfriesen: i think it would work at the qemu level but i need to dig deeper
20:55:50 jaypipes efried: say wha?
20:56:02 cfriesen sean-k-mooney[m]: unless we can do that, we can't let a single-numa-node guest use memory from multiple host numa nodes.
20:56:05 sean-k-mooney[m] jaypipes: ya i actully prefer that each group is a different RP. that is what i had originally assumed
20:56:32 jaypipes efried: I mean, I don't see a use case for doing, for example, this: resources1=VCPU:1,required2=HW_NIC_OFFLOAD_GENEVE
20:56:33 efried jaypipes: It wouldn't make sense for me to say DISK_GB:1024,VF:1. Maybe I misunderstood your statement.
20:56:40 jaypipes efried: that doesn't make sense to me.
20:57:42 sean-k-mooney[m] cfriesen: well thats the thing they are not really singel numa guests
20:57:45 efried jaypipes: resources1=VF:1,BW:200&required1=PHYSNET_A&resources2=VF:1,BW:300&required2=REALLY_FAST
20:58:06 sean-k-mooney[m] they are guest that did no specify a numa topology so there is no reason nova could not make them multi numa guests
20:58:12 jaypipes efried: ok, and?
20:58:29 cfriesen sean-k-mooney[m]: ooh, fun. that won't surprise anyone. :)
20:58:29 jaypipes efried: that makes total sense to me.
20:58:31 efried Contrived example. I don't care which physnet that second VF is on. I just want it to be fast. Why can't it land on the same RP as the first one?
20:58:56 cfriesen sean-k-mooney[m]: don't forget we'd have to preserve the numa topology over live migration
20:59:15 sean-k-mooney[m] cfriesen: if we made them multi numa guest when there resouces were split across host numa nodes it would fix the accounting issue and improve the performance of the guest
20:59:28 jaypipes efried: because it's just crazy to reason about for the user, frankly.
20:59:29 sean-k-mooney[m] cfriesen: your relying on an implentation deatil that i vift-driver specific
20:59:45 cfriesen sean-k-mooney[m]: it could also make the guest slower if it's OS isn't numa-aware.
21:00:10 jaypipes efried: if a user requests two groups of a resource, the user expects those groups to be separate.
21:00:25 jaypipes efried: and "separate" means "provided by different providers" in my and dansmith's book.
21:00:33 sean-k-mooney[m] cfriesen: the guest os
21:00:46 cfriesen sean-k-mooney[m]: yep
21:00:48 efried jaypipes: disagree completely. IMO it's way easier for the user to reason about the groups individually, without regard for their interrelations.
21:01:06 sean-k-mooney[m] cfriesen: how would it make it slow when before it would have been spread on the phyical host but it never would have knonw
21:01:43 jaypipes efried: I'm afraid we'll just have to agree to disagree.
21:01:48 efried jaypipes: Fact is, either way we decide this will leave a hole, cases we can't express.
21:01:55 cfriesen sean-k-mooney[m]: code running in the guest on two separate cpus that wants to share a lot of memory between the two CPUs. now you've got cross-NUMA latencies
21:01:56 jaypipes efried: true.
21:02:43 efried jaypipes: The way it's currently written is simpler, more flexible, and easier to implement; and IMO easier to reason about. But that may just be because it's what I've had in my head since early Q.
21:03:18 sean-k-mooney[m] cfriesen: yes and today that can happen too. the guest sees 1 numa node but on the host ist ram can be allocate form multiple because it floats
21:03:18 cfriesen sean-k-mooney[m]: you're correct that it's no worse than it is now
21:03:28 jaypipes My brain is, frankly, pretty fried.
21:03:32 efried I've thought through all the things you can't express with it, and it boils down to just one thing: you can't express "separate these request groups".
21:03:41 jaypipes I really need to eat and refresh my brain juices.
21:03:54 efried ...but adding that feature would be relatively straightforward with the addition of some new syntax.
21:04:16 efried Yeah, for my part I really ought to go review some more specs that aren't already approved.
21:06:19 sean-k-mooney[m] cfriesen: :) so i was suggesting was if you did not spcify a numa topology then we leave it up to the virt-driver but it then has to reflect the topolgy to the guest. it would be no wose then today but actully fix the 4k pages issue and technically i dont think we are breaking any guartees as i dont hink we define that if numa_nodes is not set its 1 numa node
21:07:41 sean-k-mooney[m] cfriesen: anyway that an issue for another day
21:11:22 openstackgerrit Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247
21:11:50 openstackgerrit Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971
21:12:09 edleafe cdent: efried: jaypipes: ^^ updated with your suggestions
21:14:38 efried edleafe: Check yer whitespace
21:15:46 edleafe efried: ugh, do I have to? <whine>
21:15:55 efried edleafe: But hold, there's another typo
21:16:02 efried edleafe: And another thing that needs to be said.
21:16:54 edleafe well, I wasn't gonna push a rev for whitespace until everyone had a crack at it
21:20:23 efried edleafe: Never mind that last thing. I was gonna ask if the generation in PUT /allocations/{c} was going to quit being ignored. But that's a RP generation, and not really relevant here.
21:21:04 efried edleafe: Consider me cracked.
21:22:24 sean-k-mooney[m] you know that feeling when you find the souce of a bug you hit and you wish you had not...
21:23:04 sean-k-mooney[m] here we convert form the disk bus the use asked for to a prefix https://github.com/openstack/nova/blob/ef0ce4d692d28a7f5a0079e24acdbfe7d2767e8b/nova/virt/libvirt/blockinfo.py#L124-L143
21:23:15 cdent cracked
21:23:23 sean-k-mooney[m] here we convert form the prefix to the disk bus https://github.com/openstack/nova/blob/ef0ce4d692d28a7f5a0079e24acdbfe7d2767e8b/nova/virt/libvirt/blockinfo.py#L297-L304
21:23:54 sean-k-mooney[m] we map sata scis and usb to sd then hard code sd = scsi
21:24:40 sean-k-mooney[m] that mean on kvm you can request sata or usb disk which is why i had to go to IDE to fix my centos image
21:24:40 edleafe efried: I always have
21:25:10 sean-k-mooney[m] *cant request
21:26:30 openstackgerrit Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247
21:27:20 sean-k-mooney[m] setting the disk_bus for kvm host has apreantely been broken for 5 years since https://github.com/openstack/nova/commit/7be531fe9462f2b07d4a1abf6687f649d1dfbb89 was merged ...
21:28:25 openstackgerrit Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971
21:29:18 openstackgerrit Merged openstack/nova-specs master: Amend the migration paging spec for uuid in server migrations response https://review.openstack.org/532904
21:31:01 edleafe cdent: efried: jaypipes: ^^ fixed those ghastly errors
21:31:18 efried edleafe: +1
21:35:29 mriedem jaypipes: i'll come back to the mirror aggregates in placement spec, but not today, burned out
21:37:54 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857
21:39:02 cdent thanks for putting some eyes on the error code spec jaypipes
21:41:34 openstackgerrit Sylvain Bauza proposed openstack/nova-specs master: Proposes Multiple GPU types https://review.openstack.org/557065
21:41:54 bauzas jaypipes: mriedem: just created the spec we discussed about multiple types
21:42:10 bauzas it's a very simple spec, but needs agreement
21:43:06 openstackgerrit Merged openstack/nova-specs master: Address review comments from afdc828db3c9d0205b6ded268db24f5cdf857fa6 https://review.openstack.org/554251
21:46:31 mriedem bauzas: i don't remember talking about that at all
21:46:33 melwitt bauzas: on https://blueprints.launchpad.net/nova/+spec/vgpu-rocky it looks like you can just use that bp to link with your spec since it was meant to cover the multiple vgpu types part anyway. the spec is just needed to facilitate discussion on the details, right?
21:46:40 mriedem i might have been in ireland when it happened, but didn't talk about it
21:47:50 melwitt yeah, I don't remember particulars about the vgpu discussion other than, there's more work to do this cycle and we're agreed to do it and review it
21:48:16 bauzas mriedem: no worries, I pinged you a while ago just about whether I should use another BP for it, and you told me to ping melwitt
21:48:41 bauzas melwitt: yup, zactly
21:48:46 bauzas melwitt: I linked that BP to the spec
21:48:58 melwitt okay, cool. thanks
21:49:12 bauzas so, I just want to make clear that the spec only covers the problem in it
21:49:20 bauzas which requires a consensus, hence a spec
21:49:37 bauzas other feature patches won't need it
21:49:38 melwitt right, just have to agree how to implement the multiple types
21:49:41 melwitt yeah
21:50:12 bauzas I'd love jianghuaw_ to voice on that spec too
21:54:03 bauzas anyway, calling it a day \o

Earlier   Later