| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-09 | |||
| 12:47:53 | sean-k-mooney | i just think the RT will do less work over time. | |
| 12:48:52 | alex_xu | yea, that i can imagine | |
| 12:49:39 | sean-k-mooney | it partly why i was hoping the move/rebiuld/noop/live-migration claim would eventury be removed | |
| 12:49:57 | sean-k-mooney | i was hoping we could simply what it does over time to not need them | |
| 12:51:27 | alex_xu | ok, i see | |
| 12:57:19 | sean-k-mooney | is the only place we store the vpmem allocation the in instance_extra table in the resouces filed | |
| 12:58:14 | sean-k-mooney | then in memeoy the resouce tracker discovers the avialabel vpmem namespaces and uses the instnace extra info to determin which ones are free and which are in use | |
| 13:01:01 | sean-k-mooney | ah yes we model this using the generic Resocue object and the specalised vpmemdevcie metaddata | |
| 13:01:07 | sean-k-mooney | ths is all coming back to me https://github.com/openstack/nova/blob/a4311c401b55c27321de7b282b242661af73fa34/nova/objects/resource.py#L39-L110 | |
| 13:03:34 | alex_xu | sean-k-mooney: yes | |
| 13:04:55 | sean-k-mooney | right so i rememebr we discussed if we need to do host side tracking like we do for pci device or if im memory + instastnce level tracking was enought like we do for vgpu | |
| 13:05:20 | sean-k-mooney | and we endup not doing host level tracking in the db | |
| 13:05:53 | alex_xu | yes | |
| 13:06:54 | sean-k-mooney | ya ok so the appoch in the spec makes sense then with that context. | |
| 13:07:00 | alex_xu | vgpu is more libvirt level tracking... | |
| 13:08:44 | sean-k-mooney | ya i was just looking at the vpmem discovery code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L429-L514 | |
| 13:18:08 | huaqiang | 1;2C1;5C | |
| 13:20:46 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669167 | |
| 13:21:56 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: Skip cpu comparison on AArch64 https://review.opendev.org/699116 | |
| 13:50:02 | efried | Nova meeting in 10 minutes in #openstack-meeting | |
| 13:52:16 | efried | At a glance this morning it looked to me like nova-next is failing 100%. Has anyone looked at this? | |
| 13:52:35 | efried | ...before I dig in with my non-mriedem-ness? | |
| 13:53:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add a workaround config toggle to refuse ceph image upload https://review.opendev.org/657078 | |
| 14:00:22 | efried | Nova meeting now. | |
| 14:01:55 | efried | stephenfin, sean-k-mooney, bauzas: ^ | |
| 14:02:09 | efried | huaqiang: ^ | |
| 14:02:10 | efried | who else? | |
| 14:03:39 | brinzhang_ | :) | |
| 14:06:16 | brinzhang_ | Can I send my questions? :P | |
| 14:07:04 | alex_xu | brinzhang_: meeting is at #openstack-meeting channel | |
| 14:07:25 | brinzhang_ | alex_xu: yeah, joined | |
| 14:07:29 | efried | brinzhang_: yes, for sure. | |
| 14:07:59 | brinzhang_ | alex_xu, efried: thanks | |
| 14:29:01 | openstackgerrit | Merged openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | |
| 14:30:45 | bauzas | efried: sorry was on meeting | |
| 15:01:36 | sean-k-mooney | bauzas: every time you or somone else at redhat says they are sorry i was in a meeting i go crap did i miss one or was it a 1:1 :) | |
| 15:01:54 | bauzas | 1:1 indeed | |
| 15:02:09 | sean-k-mooney | although that is partly because im getting used to meeting again after the break | |
| 15:02:15 | brinzhang_ | hah | |
| 15:02:28 | efried | I don't like that personally because I think the mix of placement-ese and hw:-ese is more confusing than just hw:-ese | |
| 15:02:28 | efried | We could make that work, if we had to, because the hw:-ese part could impart enough information that you could infer the rest about your shared CPUs. | |
| 15:02:28 | efried | dansmith: Re specifying VCPUs in placement-ese and other stuff in hw:-ese... | |
| 15:02:58 | efried | point being: we're going to have to have snowflake syntax anyway, so might as well make it consistent. | |
| 15:03:36 | sean-k-mooney | efried: you coudld infer the amoung and then have a driver specifc convetion for how to select which ones would float and whcih woudl be pinned | |
| 15:03:44 | efried | Rather than asking people to understand which subset of placement-ese is side-effecty and which is not | |
| 15:03:45 | sean-k-mooney | but it make the code more complex | |
| 15:04:03 | efried | sean-k-mooney: Yes, exactly. Yes, exactly. | |
| 15:04:16 | efried | and also make it harder for the user to understand IMO. | |
| 15:05:36 | sean-k-mooney | if i was to code the algortiom it would be balance the shared cores evenly across numa nodes then make the rest pinned | |
| 15:05:40 | dansmith | I dunno, I'm just disappointed with the mess we've had, do have, and apparently will have | |
| 15:06:22 | sean-k-mooney | which in a singel numa node guest means all the shared cores are fist follow by the pinned cores | |
| 15:06:50 | dansmith | do the public clouds let you have this level of control over which cpus are pinned and to where? | |
| 15:06:54 | sean-k-mooney | and that is because the linux kernel and proably other has a prefernce for runing some code on core 0 of each numa node | |
| 15:07:38 | efried | sean-k-mooney: even allowing for future expansion where you can specify how many you want in each NUMA node, we could still do the math. My gripe is really about the UX. | |
| 15:08:14 | efried | dansmith: is that a leading question? | |
| 15:08:46 | sean-k-mooney | well today the just expose flaver with either pinning on or off or numa nodes. but you can configure this via the image too | |
| 15:08:52 | dansmith | efried: it's an honest one.. we used to have a shared value of "abstraction means not getting to turn every knob" | |
| 15:09:08 | efried | Isn't "our competitor can't do X, and neither can we, so let's make X possible so we're better than our competitor" how progress is made? | |
| 15:09:35 | efried | dansmith: and that abstraction still exists. You can still put up a flavor with no extra specs and get a VM. We're not taking away that simplicity. | |
| 15:09:57 | dansmith | efried: the same argument goes fo "some vendor shows up with a P2V'd ancient appliance that requires a specific hardware topology and we spent years trying to implement that instead of actually competing on something that matters" | |
| 15:10:04 | sean-k-mooney | so as an operate i can create a flavor with 8 cores and pinnnge and then as a user i can upload an image the says make the vm have 1 socket and, 2 thread per core, and 1 nuam node please | |
| 15:10:25 | efried | dansmith: I dig. But NUMA isn't going away. | |
| 15:10:52 | efried | and edge tuning is like the big thing, isn't it? | |
| 15:11:01 | dansmith | efried: no, numa isn't, that's not the point.. whether or not we need to provide this level of control over what the topology looks like instead of just providing a sane one with the asks we were given is. | |
| 15:11:26 | dansmith | efried: mostly because of legacy appliances afaik :) | |
| 15:11:32 | efried | if a good-enough VM was good enough, sure. | |
| 15:12:01 | efried | but in edge IIUC you need to be able to squeeze every cycle out of your hardware, no? | |
| 15:12:15 | sean-k-mooney | well if we were to provide a sane topology by default we would have to change our curren policy of 1 cpu socket per vcpu | |
| 15:12:25 | sean-k-mooney | but i get what you mean | |
| 15:12:27 | dansmith | that's not generally the reason they need to strictly control the topology.. it seems like the reason, but it's often not :) | |
| 15:12:33 | sean-k-mooney | sane default would be nice | |
| 15:12:48 | efried | I think we have sane defaults, don't we? | |
| 15:12:49 | dansmith | anyway, I'm not trying to stop the train on those grounds, | |
| 15:13:10 | sean-k-mooney | efried: i would not call the cpu toplogy nova generates sane | |
| 15:13:15 | efried | heh | |
| 15:13:22 | efried | well, it could be | |
| 15:13:34 | dansmith | I'm more looking for a signpost in the distance of what level of abstraction others have chosen | |
| 15:13:41 | sean-k-mooney | if you have a 16 VCPU guest it will give you a guest with 16 cpu socket each wtih a 1 core 1 thread cpu | |
| 15:14:11 | sean-k-mooney | and for software licence that chages per socket that is expensive | |
| 15:14:27 | sean-k-mooney | that iw why the cpu topology extra specs where added | |
| 15:14:35 | dansmith | exactly :) | |
| 15:14:40 | dansmith | not for perf tuning :) | |
| 15:14:45 | sean-k-mooney | windows server used to be changed per socket not per core | |
| 15:15:04 | dansmith | and nvidia has some similar things | |
| 15:15:16 | sean-k-mooney | so if you are sane you would do hw:cpu_socket=1 hw:cpu_theads=2 | |
| 15:15:19 | dansmith | and closed source P2V'd ancient telco appliances are far far worse | |
| 15:15:24 | efried | dansmith: I think I see your point. It should be *possible* to say "give me $p pinned and $s shared, you go figure out how to distribute them" without *having* to say "I want CPUs 0,3,7 pinned and 1,2,4-6 shared". | |
| 15:15:44 | dansmith | efried: pretty much | |
| 15:15:58 | sean-k-mooney | efried: yes so that is what we wanted to discuss in the meeting when we said count vs mask | |
| 15:16:01 | efried | But I would argue that, if you care about a pinned vs shared split, you care enough about performance that having to do the extra syntax isn't a big jump. | |
| 15:16:04 | dansmith | there's also the device-to-numa affinity which I understand is important, so that your nic is affined to your dedicated cpus | |
| 15:16:12 | efried | mm | |
| 15:16:48 | sean-k-mooney | dansmith: that is out of scope of this spec but yes that is a factor | |
| 15:17:06 | dansmith | certainly you know I'm not arguing that operators of multi-billion-dollar clouds shouldn't have to write 255 characters of ascii syntax for this right? | |
| 15:17:22 | dansmith | sean-k-mooney: no, I know, I was just commenting that I know asking for the abstract isn't enough | |
| 15:17:30 | dansmith | sean-k-mooney: or rather, asking for his abstract example | |
| 15:17:50 | sean-k-mooney | so if we take numa as an example | |
| 15:17:55 | sean-k-mooney | we actully support both | |
| 15:18:04 | sean-k-mooney | if you do hw:numa_nodes=2 | |
| 15:18:17 | sean-k-mooney | and say nothing esle we devied the cpu and ram evenly between the numa nodes | |