| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-09 | |||
| 13:01:07 | sean-k-mooney | ths is all coming back to me https://github.com/openstack/nova/blob/a4311c401b55c27321de7b282b242661af73fa34/nova/objects/resource.py#L39-L110 | |
| 13:03:34 | alex_xu | sean-k-mooney: yes | |
| 13:04:55 | sean-k-mooney | right so i rememebr we discussed if we need to do host side tracking like we do for pci device or if im memory + instastnce level tracking was enought like we do for vgpu | |
| 13:05:20 | sean-k-mooney | and we endup not doing host level tracking in the db | |
| 13:05:53 | alex_xu | yes | |
| 13:06:54 | sean-k-mooney | ya ok so the appoch in the spec makes sense then with that context. | |
| 13:07:00 | alex_xu | vgpu is more libvirt level tracking... | |
| 13:08:44 | sean-k-mooney | ya i was just looking at the vpmem discovery code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L429-L514 | |
| 13:18:08 | huaqiang | 1;2C1;5C | |
| 13:20:46 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669167 | |
| 13:21:56 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: Skip cpu comparison on AArch64 https://review.opendev.org/699116 | |
| 13:50:02 | efried | Nova meeting in 10 minutes in #openstack-meeting | |
| 13:52:16 | efried | At a glance this morning it looked to me like nova-next is failing 100%. Has anyone looked at this? | |
| 13:52:35 | efried | ...before I dig in with my non-mriedem-ness? | |
| 13:53:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add a workaround config toggle to refuse ceph image upload https://review.opendev.org/657078 | |
| 14:00:22 | efried | Nova meeting now. | |
| 14:01:55 | efried | stephenfin, sean-k-mooney, bauzas: ^ | |
| 14:02:09 | efried | huaqiang: ^ | |
| 14:02:10 | efried | who else? | |
| 14:03:39 | brinzhang_ | :) | |
| 14:06:16 | brinzhang_ | Can I send my questions? :P | |
| 14:07:04 | alex_xu | brinzhang_: meeting is at #openstack-meeting channel | |
| 14:07:25 | brinzhang_ | alex_xu: yeah, joined | |
| 14:07:29 | efried | brinzhang_: yes, for sure. | |
| 14:07:59 | brinzhang_ | alex_xu, efried: thanks | |
| 14:29:01 | openstackgerrit | Merged openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | |
| 14:30:45 | bauzas | efried: sorry was on meeting | |
| 15:01:36 | sean-k-mooney | bauzas: every time you or somone else at redhat says they are sorry i was in a meeting i go crap did i miss one or was it a 1:1 :) | |
| 15:01:54 | bauzas | 1:1 indeed | |
| 15:02:09 | sean-k-mooney | although that is partly because im getting used to meeting again after the break | |
| 15:02:15 | brinzhang_ | hah | |
| 15:02:28 | efried | I don't like that personally because I think the mix of placement-ese and hw:-ese is more confusing than just hw:-ese | |
| 15:02:28 | efried | We could make that work, if we had to, because the hw:-ese part could impart enough information that you could infer the rest about your shared CPUs. | |
| 15:02:28 | efried | dansmith: Re specifying VCPUs in placement-ese and other stuff in hw:-ese... | |
| 15:02:58 | efried | point being: we're going to have to have snowflake syntax anyway, so might as well make it consistent. | |
| 15:03:36 | sean-k-mooney | efried: you coudld infer the amoung and then have a driver specifc convetion for how to select which ones would float and whcih woudl be pinned | |
| 15:03:44 | efried | Rather than asking people to understand which subset of placement-ese is side-effecty and which is not | |
| 15:03:45 | sean-k-mooney | but it make the code more complex | |
| 15:04:03 | efried | sean-k-mooney: Yes, exactly. Yes, exactly. | |
| 15:04:16 | efried | and also make it harder for the user to understand IMO. | |
| 15:05:36 | sean-k-mooney | if i was to code the algortiom it would be balance the shared cores evenly across numa nodes then make the rest pinned | |
| 15:05:40 | dansmith | I dunno, I'm just disappointed with the mess we've had, do have, and apparently will have | |
| 15:06:22 | sean-k-mooney | which in a singel numa node guest means all the shared cores are fist follow by the pinned cores | |
| 15:06:50 | dansmith | do the public clouds let you have this level of control over which cpus are pinned and to where? | |
| 15:06:54 | sean-k-mooney | and that is because the linux kernel and proably other has a prefernce for runing some code on core 0 of each numa node | |
| 15:07:38 | efried | sean-k-mooney: even allowing for future expansion where you can specify how many you want in each NUMA node, we could still do the math. My gripe is really about the UX. | |
| 15:08:14 | efried | dansmith: is that a leading question? | |
| 15:08:46 | sean-k-mooney | well today the just expose flaver with either pinning on or off or numa nodes. but you can configure this via the image too | |
| 15:08:52 | dansmith | efried: it's an honest one.. we used to have a shared value of "abstraction means not getting to turn every knob" | |
| 15:09:08 | efried | Isn't "our competitor can't do X, and neither can we, so let's make X possible so we're better than our competitor" how progress is made? | |
| 15:09:35 | efried | dansmith: and that abstraction still exists. You can still put up a flavor with no extra specs and get a VM. We're not taking away that simplicity. | |
| 15:09:57 | dansmith | efried: the same argument goes fo "some vendor shows up with a P2V'd ancient appliance that requires a specific hardware topology and we spent years trying to implement that instead of actually competing on something that matters" | |
| 15:10:04 | sean-k-mooney | so as an operate i can create a flavor with 8 cores and pinnnge and then as a user i can upload an image the says make the vm have 1 socket and, 2 thread per core, and 1 nuam node please | |
| 15:10:25 | efried | dansmith: I dig. But NUMA isn't going away. | |
| 15:10:52 | efried | and edge tuning is like the big thing, isn't it? | |
| 15:11:01 | dansmith | efried: no, numa isn't, that's not the point.. whether or not we need to provide this level of control over what the topology looks like instead of just providing a sane one with the asks we were given is. | |
| 15:11:26 | dansmith | efried: mostly because of legacy appliances afaik :) | |
| 15:11:32 | efried | if a good-enough VM was good enough, sure. | |
| 15:12:01 | efried | but in edge IIUC you need to be able to squeeze every cycle out of your hardware, no? | |
| 15:12:15 | sean-k-mooney | well if we were to provide a sane topology by default we would have to change our curren policy of 1 cpu socket per vcpu | |
| 15:12:25 | sean-k-mooney | but i get what you mean | |
| 15:12:27 | dansmith | that's not generally the reason they need to strictly control the topology.. it seems like the reason, but it's often not :) | |
| 15:12:33 | sean-k-mooney | sane default would be nice | |
| 15:12:48 | efried | I think we have sane defaults, don't we? | |
| 15:12:49 | dansmith | anyway, I'm not trying to stop the train on those grounds, | |
| 15:13:10 | sean-k-mooney | efried: i would not call the cpu toplogy nova generates sane | |
| 15:13:15 | efried | heh | |
| 15:13:22 | efried | well, it could be | |
| 15:13:34 | dansmith | I'm more looking for a signpost in the distance of what level of abstraction others have chosen | |
| 15:13:41 | sean-k-mooney | if you have a 16 VCPU guest it will give you a guest with 16 cpu socket each wtih a 1 core 1 thread cpu | |
| 15:14:11 | sean-k-mooney | and for software licence that chages per socket that is expensive | |
| 15:14:27 | sean-k-mooney | that iw why the cpu topology extra specs where added | |
| 15:14:35 | dansmith | exactly :) | |
| 15:14:40 | dansmith | not for perf tuning :) | |
| 15:14:45 | sean-k-mooney | windows server used to be changed per socket not per core | |
| 15:15:04 | dansmith | and nvidia has some similar things | |
| 15:15:16 | sean-k-mooney | so if you are sane you would do hw:cpu_socket=1 hw:cpu_theads=2 | |
| 15:15:19 | dansmith | and closed source P2V'd ancient telco appliances are far far worse | |
| 15:15:24 | efried | dansmith: I think I see your point. It should be *possible* to say "give me $p pinned and $s shared, you go figure out how to distribute them" without *having* to say "I want CPUs 0,3,7 pinned and 1,2,4-6 shared". | |
| 15:15:44 | dansmith | efried: pretty much | |
| 15:15:58 | sean-k-mooney | efried: yes so that is what we wanted to discuss in the meeting when we said count vs mask | |
| 15:16:01 | efried | But I would argue that, if you care about a pinned vs shared split, you care enough about performance that having to do the extra syntax isn't a big jump. | |
| 15:16:04 | dansmith | there's also the device-to-numa affinity which I understand is important, so that your nic is affined to your dedicated cpus | |
| 15:16:12 | efried | mm | |
| 15:16:48 | sean-k-mooney | dansmith: that is out of scope of this spec but yes that is a factor | |
| 15:17:06 | dansmith | certainly you know I'm not arguing that operators of multi-billion-dollar clouds shouldn't have to write 255 characters of ascii syntax for this right? | |
| 15:17:22 | dansmith | sean-k-mooney: no, I know, I was just commenting that I know asking for the abstract isn't enough | |
| 15:17:30 | dansmith | sean-k-mooney: or rather, asking for his abstract example | |
| 15:17:50 | sean-k-mooney | so if we take numa as an example | |
| 15:17:55 | sean-k-mooney | we actully support both | |
| 15:18:04 | sean-k-mooney | if you do hw:numa_nodes=2 | |
| 15:18:17 | sean-k-mooney | and say nothing esle we devied the cpu and ram evenly between the numa nodes | |
| 15:18:49 | sean-k-mooney | but you can also use hw:numa_cpu.0=2 .... to speciy the numa of cpus or ram per numa node | |
| 15:19:04 | sean-k-mooney | most peopel jsut hw:numa_nodes and let nova do it | |
| 15:19:58 | sean-k-mooney | i guess the diffferen with numa id they can detect the cpu and ram in the gues via the kernel jsut as if it was real hardware | |
| 15:20:40 | sean-k-mooney | pining vs floaing is a qualtive change ranter then quantative so if they care about that then need another openstack sepcifc way to determin that | |
| 15:22:04 | sean-k-mooney | the primary use case for this by the way that im aware of is for realtime guests where they want all the realtime cores to be pinned and the non realtime managment core to be floating so that they can get the most desity out of a host | |
| 15:22:08 | dansmith | efried: so I guess to close, | |
| 15:23:18 | dansmith | efried: my preference (and what I thought we were moving towards) was a future where resource (amounts) are specified in placement syntax, and topologies and the like are assigned sanely. Tweaks to how those look (like I really need this thing pinned here) would be hw:things but where the hw: is the optional | |
| 15:23:52 | dansmith | meaning, usually you just ask for the amount of stuff you want and we give it to you in a sane arrangement which is fine for most people, | |