| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-07 | |||
| 15:29:37 | sean-k-mooney | but its quite common to only have nics attached to numa 0 | |
| 15:29:39 | efried | It's not? I thought that was the point of saying hw:numa_nodes=2 | |
| 15:29:51 | sean-k-mooney | no | |
| 15:29:54 | efried | Sorry, I mean if the CPU/mem resources in the NUMA nodes are symmetric | |
| 15:30:05 | efried | I don't mean "same number of VFs per NUMA node" | |
| 15:30:05 | sean-k-mooney | hw:numa_nodes=2 is refering to the guest numa toplogy | |
| 15:30:19 | sean-k-mooney | oh by default yest | |
| 15:30:22 | sean-k-mooney | *yes | |
| 15:30:28 | efried | when we do that we're dividing evenly, yah? | |
| 15:31:00 | sean-k-mooney | you can change that with hw:numa_mem.0=512 hw:numa_mem.1=1536 | |
| 15:31:02 | huaqiang | sean-k-mooney: hi | |
| 15:31:20 | efried | sean-k-mooney: Yes, exactly, that's my point: those cases are rarer, right? | |
| 15:31:23 | sean-k-mooney | efried: yes if you jsut do hw:numa_nodes it devices cpus and ram evenly | |
| 15:31:36 | sean-k-mooney | we also check its evenly deivisable | |
| 15:31:50 | efried | sean-k-mooney: Do we currently support VF NUMA affinity? | |
| 15:31:56 | sean-k-mooney | efried: yes | |
| 15:32:06 | sean-k-mooney | for 4+ years | |
| 15:32:29 | sean-k-mooney | we require it in the case of neutron sriov which is what im trying to chagne | |
| 15:32:51 | sean-k-mooney | with https://review.opendev.org/#/c/674072/ | |
| 15:33:05 | sean-k-mooney | gibi: ^ i would like your input on that by the way | |
| 15:33:26 | sean-k-mooney | huaqiang: hi | |
| 15:33:42 | efried | sean-k-mooney: even in the case of lopsided NUMA nodes? You can say which one you want your VF associated with? | |
| 15:33:52 | sean-k-mooney | efried: no | |
| 15:33:58 | sean-k-mooney | you cannot specify that at all | |
| 15:34:00 | efried | okay, then we're fine. | |
| 15:34:08 | efried | we continue to say you can't specify that. | |
| 15:34:16 | sean-k-mooney | we guarentee it will come form one of the numa ndoes the vm is mapped too | |
| 15:34:37 | sean-k-mooney | but we give no garuntee it on a specific guest numa node | |
| 15:34:48 | gibi | efried: if nova keeps handling the VFs then nova can do a VF-NUMA affinity. Bandwidth is already tied to VF today in case of SRIOV. | |
| 15:35:02 | sean-k-mooney | currently its always preseneted as if its attache to guest numa node 0 regradelss of what host numa node it came from | |
| 15:35:05 | efried | and we can give the requests for NUMA nodes and VFs generic known suffixes, and use same_subtree to tie them together. | |
| 15:35:16 | efried | perfect | |
| 15:35:20 | sean-k-mooney | i also want to chagne that but not in a way tha will impact placmenet or shcudleing in any way | |
| 15:35:40 | efried | Yeah, let's not give any more granular control over that | |
| 15:35:47 | efried | cause then we would have to support it via placement :) | |
| 15:36:06 | efried | Okay, it's the middle of the night for huaqiang, let's wrap up and yield the floor... | |
| 15:36:17 | sean-k-mooney | efried: what i want to add i the use of a livirt feature that allows mupliple pci roots in the guest | |
| 15:36:34 | huaqiang | sean-k-mooney: got the comments for the 'using PCPU VCPU in one instance', thanks | |
| 15:36:52 | sean-k-mooney | that will allwo us to attach the pci device to the virtual numa node that maps to the host numa node the deive is on | |
| 15:37:16 | huaqiang | but I checked the existing logic for 'hw:cpu_policy=dedicated' and 'hw_cpu_policy=shared', the final policy is 'dedicated' | |
| 15:38:06 | sean-k-mooney | can you show me that in code | |
| 15:38:18 | huaqiang | sure | |
| 15:39:27 | sean-k-mooney | its this https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1327-L1338 | |
| 15:39:55 | sean-k-mooney | ok we do perfer the flavor over the image in that case | |
| 15:40:11 | huaqiang | you find it! | |
| 15:40:28 | sean-k-mooney | ya i knew what code shoudl be enforcing it | |
| 15:40:42 | sean-k-mooney | its incositent with how we handel hw:cpu_threads_policy | |
| 15:40:44 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1370-L1377 | |
| 15:41:09 | sean-k-mooney | and numa topligies https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1290 | |
| 15:41:27 | sean-k-mooney | had thought we had made it an error for all conflitc at this point | |
| 15:42:07 | sean-k-mooney | we can chagne that without at least 1 cycle to deprecate the old behavior so i guess we have to stick with the current behavior | |
| 15:42:58 | sean-k-mooney | huaqiang: it would be good to add a warning here https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1328 as i dont think that is what people expect | |
| 15:43:12 | huaqiang | so you mean the flavor first behavior | |
| 15:43:17 | sean-k-mooney | yes | |
| 15:43:41 | sean-k-mooney | we started moving to raising exception if they conflict for most of these value a whiel ago | |
| 15:43:55 | sean-k-mooney | at least when adding new values | |
| 15:44:27 | sean-k-mooney | i guess we did not change older values proably due to upgrade concerns | |
| 15:45:23 | huaqiang | so for the new 'mixed' policy, we need consider conflict as invalid | |
| 15:45:53 | huaqiang | but for 'dedicated' and 'shared' conflicts we follow the rule that flavor first | |
| 15:45:54 | sean-k-mooney | well we have a choice | |
| 15:46:31 | sean-k-mooney | my perference would be to consider all conflcit of hw:cpu_policy and hw_cpu_policy to be invalid and raise an exception | |
| 15:46:37 | sean-k-mooney | but since we cant do that htis cycle | |
| 15:46:44 | sean-k-mooney | its better to be consistent i think | |
| 15:47:06 | sean-k-mooney | it would be nice to issue a warning if we detect a conflict however | |
| 15:47:10 | huaqiang | I am good for both | |
| 15:47:37 | huaqiang | ok | |
| 15:48:04 | huaqiang | another thing i don't know if you noticed that | |
| 15:48:19 | huaqiang | hw:cpu_realtime exists | |
| 15:48:37 | huaqiang | but there is no 'hw_cpu_realtime' in image properties | |
| 15:48:59 | huaqiang | this is different with the mask for dedicated cpus | |
| 15:49:41 | huaqiang | Alex mentioned this difference, and cpu realtime mask has a more loose rules | |
| 15:50:23 | sean-k-mooney | yes we did not want to allow users to request realtime guest via the image | |
| 15:50:33 | sean-k-mooney | but we did allow the mask i belive | |
| 15:50:48 | huaqiang | mask is allowed from both image and flavor | |
| 15:51:01 | sean-k-mooney | so you can only enabel realtime cpus via the flavor but both the flavor and image can set the mask | |
| 15:51:15 | huaqiang | that is the truth | |
| 15:52:01 | sean-k-mooney | we have allowed image to enable cpu pinning provided the the flavor has not forbiding it by setting hw:cpu_policy=shared | |
| 15:52:35 | sean-k-mooney | so i think its fine wto allow hw_cpu_policy=mixed give hw_cpu_policy=dedicated is alredy allowed | |
| 15:52:55 | sean-k-mooney | similarly i think having the mask in the image and flavor is ok | |
| 15:53:11 | mkrai | Hi Nova team, I am working on implementing the resize API for baremetals and need some help in the design. I tried to put an initial design here https://etherpad.openstack.org/p/nova-resize-ironic-api | |
| 15:53:20 | sean-k-mooney | but im sure someone else has an opipion on that ^ its what the spec currently say however | |
| 15:54:15 | gibi | sean-k-mooney: left comment in https://review.opendev.org/#/c/674072 about the notification tests | |
| 15:54:16 | huaqiang | yes, dansmith is questioned on the masks | |
| 15:54:58 | sean-k-mooney | gibi: thanks i can never rememebr where that is | |
| 15:55:18 | mkrai | efried, sean-k-mooney mriedem alex_xu could you please take a look? | |
| 15:55:21 | sean-k-mooney | im fixing other typos but was about to spend the next hour finding it | |
| 15:55:24 | efried | mkrai: Hi there. I recall mriedem had some ideas about that design. dansmith could probably provide some good input as well. To be clear, this is going to be a U release thing, you're just looking for initial design help? | |
| 15:55:35 | sean-k-mooney | mkrai: yes i have it open | |
| 15:55:37 | gibi | sean-k-mooney: we might improve the test to dump the file path when the assert fails | |
| 15:55:56 | mkrai | efried, Yes! | |
| 15:56:02 | mkrai | sean-k-mooney, Thanks! | |
| 15:56:22 | huaqiang | dansmith: hello | |
| 15:58:06 | sean-k-mooney | efried: mkrai could start on any ironic internal prep work that was need this cycle but the ironic driver changes would be next cycle i guess | |
| 15:59:16 | mriedem | i love being continually distracted about asking to review specs and plans and code for things that do not even have a chance for train | |
| 15:59:34 | mkrai | sean-k-mooney, I have already posted spec for the ironic reconfigure API that will be needed by the resize API | |
| 15:59:36 | sean-k-mooney | gibi: are you ok with https://review.opendev.org/#/c/674072 adressing bug: #179520 as a bugfix to the original spec | |
| 15:59:36 | openstack | bug 179468 in python-cups (Ubuntu) "duplicate for #179520 applet.py crashed with TypeError in on_job_hold_activate()" [High,Fix released] https://launchpad.net/bugs/179468 - Assigned to Pascal De Vuyst (pascal-devuyst) | |
| 15:59:36 | mriedem | meanwhile my mega cross-cell resize series is not getting review but has been up for 2 cycles | |
| 16:00:17 | mkrai | sean-k-mooney, https://review.opendev.org/#/c/672252/ | |
| 16:00:29 | mkrai | I will add this link to the etherpad | |