| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-19 | |||
| 14:44:08 | sean-k-mooney | hw_numa_nodes=2 with hw_pci_numa_affinity_policy=require means the pci device will be on the same numa node as one fo the guest numa nodes | |
| 14:44:34 | sean-k-mooney | each device can be colacate with a different one fo the guests numa nodes | |
| 14:44:39 | artom | sean-k-mooney, I get that, because being on the same host numa node as *both* guest NUMA nodes doens't make sense | |
| 14:44:50 | sean-k-mooney | correct | |
| 14:44:57 | sean-k-mooney | well ig could | |
| 14:44:59 | artom | But being on the same *socket* as both guest NUMA nodes *would* make sense | |
| 14:45:09 | artom | So it's a matter of - what's the most useful? | |
| 14:45:14 | sean-k-mooney | we currenlty dont allow mulitple guest numa nodes to map to one host numa node | |
| 14:45:19 | sean-k-mooney | that is not a guarnette fo the api | |
| 14:45:26 | sean-k-mooney | its an implemantion detail fo the libvirt driver | |
| 14:45:41 | sean-k-mooney | one which strictly speaking operators should not rely on | |
| 14:46:03 | artom | But de facto they do, because nothing except the libvirt driver will ever implement this :P | |
| 14:46:09 | artom | Anways, that's tangential | |
| 14:46:21 | sean-k-mooney | well i have wanted to change that sicne we added numa affinity | |
| 14:46:28 | sean-k-mooney | the intel implematnion did not require this | |
| 14:47:05 | sean-k-mooney | if we want to provide a way for all nuam nodes to be confiend to the same socket | |
| 14:47:14 | sean-k-mooney | i think that is a different feature to pci affintiy | |
| 14:47:39 | artom | Ah, now that does make sense | |
| 14:47:44 | sean-k-mooney | that is more a kin to hw:numa_mem_policy which was never impelmented | |
| 14:48:16 | artom | I guess? But yeah, I see the point - forcing all guest numa nodes to come from the same host socket is a different thing from PCI policy, and you could have both | |
| 14:48:31 | sean-k-mooney | yes | |
| 14:49:13 | sean-k-mooney | so i would personally find it suprising if the pci policy would trigger that behavior | |
| 14:49:28 | sean-k-mooney | espcially coupled with the ablity to set the policy via a neutorn port | |
| 14:52:39 | artom | Fair points, lemme update | |
| 14:56:19 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/train: Use cell targeted context to query BDMs for metadata https://review.opendev.org/c/openstack/nova/+/771407 | |
| 15:02:14 | dcapone2004 | I am having an issue on ussuri where guest centos VMs with SELinux disabled or set to permissive have the systemd process in the VM consume 100% CPU and never goes back down without rebooting the VM | |
| 15:02:18 | dcapone2004 | after rebooting, the issue recurs within several hours | |
| 15:03:46 | dcapone2004 | I have been attempting to google for other reports of this issue and have been unable to find anything and was trying to see if anyone has seen this issue before and knows what might cause it or any type of resolution | |
| 15:05:02 | dcapone2004 | strace on the systemd process in these VMs, just shows a bunch of epoll_pwait like systemd is hung waiting for something | |
| 15:09:00 | openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: libvirt: Add support for flavor and image defined ephemeral encryption https://review.opendev.org/c/openstack/nova-specs/+/771474 | |
| 15:09:51 | sean-k-mooney | artom: finished with the review | |
| 15:12:01 | artom | sean-k-mooney, cheers! | |
| 15:12:09 | artom | Hrmm, yeah, placement trait would be faster... | |
| 15:12:34 | artom | I just kinda dislike that it's becoming this endless stream of "NEW_FEATURE_TRAIT_TO_HANDLE_ROLLING_UPGRADES" | |
| 15:12:39 | artom | But maybe that's what they're for... | |
| 15:15:11 | sean-k-mooney | well that was basically the entire point of traits | |
| 15:15:20 | sean-k-mooney | so we can expose capabityes and schdule on them | |
| 15:15:42 | sean-k-mooney | although maybe not for you example | |
| 15:15:53 | sean-k-mooney | the feature trait is not actully for rolling upgrdes | |
| 15:16:08 | sean-k-mooney | it can help with that but its for spawn and all move operations | |
| 15:16:19 | sean-k-mooney | so that we would only consider host that support socket affinity | |
| 15:16:30 | sean-k-mooney | which is very useful in a mixed hypervior deployment | |
| 15:28:28 | artom | Note to self, when talking about dicts flying over the wire, do not typo "wire" as "wife" | |
| 15:45:44 | lyarwood | oh my | |
| 15:48:03 | artom | (The oh my) | |
| 15:48:03 | artom | Reminds me of https://www.youtube.com/watch?v=XE1XE6Thp9E for some reason | |
| 15:48:30 | lyarwood | risky click of the day | |
| 15:48:34 | lyarwood | oh that part | |
| 15:48:36 | lyarwood | lol | |
| 15:52:39 | sean-k-mooney | lol | |
| 15:52:48 | sean-k-mooney | why | |
| 16:05:36 | openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: `socket` PCI NUMA-affinity Policy https://review.opendev.org/c/openstack/nova-specs/+/765551 | |
| 16:06:36 | artom | sean-k-mooney, ^^ I'll look at the vdpa spec now. I think the only thing I had a bit of a beef with is the whitelist. IIUC, it would have to contian the parent PF of any vdpa devices, so that should get documented methinks | |
| 16:36:22 | efried | I've never used "ta". | |
| 16:36:39 | efried | I did grow up mostly in London, so I wasn't a stranger to the idiom. But it never made it into my lexicon. | |
| 16:37:32 | sean-k-mooney | efried: really never knew you had a london connection | |
| 16:37:45 | sean-k-mooney | efried: hows life in k8s land | |
| 16:44:36 | sean-k-mooney | artom: +1 on your spec. i think that is consitent with my expections now | |
| 17:01:39 | artom | sean-k-mooney, cool, thanks! Yeah, I dunno if the "parent PF must be whitelisted" thing is obvious, but I figure it'd be nice to document it for operators | |
| 17:02:02 | artom | I think what confused me is you initially said in a review comment that the whitelist had nothing to do with VDPA | |
| 17:05:54 | sean-k-mooney | well the PF should not nessisaryly be what you whitelist it could be but you could whitelist the VFs | |
| 17:06:23 | sean-k-mooney | the imporant part is we are not changing how the whitelist currenlty works | |
| 17:07:16 | sean-k-mooney | it declare which pci device can be passthough, we then check if any of those have vdpa devices assocated with them and if they do we mark them as type-VDPA | |
| 17:07:28 | sean-k-mooney | instead of type-VF | |
| 17:07:58 | sean-k-mooney | that is part o fthe code i still need to merge | |
| 17:08:03 | sean-k-mooney | *write | |
| 17:08:27 | efried | OpenShift is fun. I'm not loving the devops side of things, but it's tolerable. | |
| 17:08:27 | efried | sean-k-mooney: Yeah, lived just outside of London from 84-87 and 89-94. | |
| 17:09:02 | sean-k-mooney | efried: have you managed to fine a way to do upstream work or still more on the support side | |
| 17:09:18 | efried | For some value of "upstream". | |
| 17:09:43 | sean-k-mooney | well i basicaly ment coding/bugfixing/feature design | |
| 17:09:50 | efried | Yeah, I'm getting to write code. | |
| 17:10:06 | sean-k-mooney | that good that they are not wasting your experince then at least | |
| 17:10:10 | efried | This is my baby: https://github.com/openshift/boilerplate/ | |
| 17:10:39 | sean-k-mooney | kind of like cookiecutter? | |
| 17:11:39 | sean-k-mooney | efried: ah not quite | |
| 17:11:48 | efried | Similar spirit. Doesn't cookiecutter play just at the birth of a repo? Boilerplate continues to evolve and subscribers get updates as they go. | |
| 17:11:53 | sean-k-mooney | efried: its more vendorisizng the standard deps | |
| 17:12:12 | efried | not deps in the package sense. | |
| 17:12:35 | efried | But yeah, kinda like vendoring standard workflows, make targets, build scripts... | |
| 17:12:37 | sean-k-mooney | well you use the term artifacts | |
| 17:12:49 | sean-k-mooney | so im thinking of them as a stanard libviar of things you can use | |
| 17:14:12 | efried | Yeah, something like that. Except the default is *must* use; we make it hard to switch pieces off. | |
| 17:15:12 | sean-k-mooney | presumably for consitncy between different operators | |
| 17:15:38 | sean-k-mooney | and preventing code debt thorugh duplciation | |
| 17:15:43 | efried | exactly. | |
| 17:16:13 | artom | sean-k-mooney, right, so I think that's the part that could be made more explicit (at least for my brain): "the existing whitelist mechanism is unchanged - if a device is whitelisted and is bound to the vdpa driver, it will be inventoried as type-VDPA and plugged accordingly." | |
| 17:16:58 | sean-k-mooney | artom: sure i can adde that proably in the developer inpact section | |
| 17:17:27 | sean-k-mooney | * Other deployer impact | |
| 17:17:33 | sean-k-mooney | sorry no | |
| 17:17:38 | sean-k-mooney | Developer impact | |
| 17:17:53 | sean-k-mooney | ill add it now in any case | |
| 17:19:51 | sean-k-mooney | i guess technically im modifying this code https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L7074-L7110 so i can put it in proposed change section | |
| 17:37:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach https://review.opendev.org/c/openstack/nova/+/770246 | |
| 17:52:03 | dansmith | gmann: hey I was looking for some fat to cut from jobs.. just thinking.. do we really need to run grenade and grenade-multinode against nova? not sure under what circumstances we'd really get benefit from regular grenade that we don't see in the multinode job | |
| 17:55:09 | gmann | dansmith: multinode you mean this right ( not grenade-multinode ) nova-grenade-multinode ? | |
| 17:55:21 | dansmith | sure | |
| 17:56:00 | gmann | dansmith: it run live migration tests too https://github.com/openstack/nova/blob/master/playbooks/legacy/nova-grenade-multinode/run.yaml#L58 | |
| 17:56:39 | dansmith | gmann: right, but is there anything fundamental about the base grenade job that we don't or can't do in the multinode variant such that we could stop running the base grenade job? | |
| 17:56:52 | gmann | dansmith: or you are thinking to drop grenade-py3 in favor of nova-grenade-multinode which cover upgrade + live migration? | |