| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-10-24 | |||
| 18:27:46 | dansmith | idk, ages ago (like icehouse timeframe) when we were transitioning from the mysql C client to the python-native one, I thought that we encountered a situation where local file i/o was not patched and that surprised me, and some of the elders explained that it wasn't in scope | |
| 18:28:30 | sean-k-mooney | i mean we can disabel patchign of this | |
| 18:28:36 | dansmith | but that was a long time ago and we've shuffled our own monkey patching a lot since then, so maybe we weren't patching os at the time, or maybe I'm just misremembering | |
| 18:28:40 | sean-k-mooney | we coudl set os=false adn builtins=false | |
| 18:29:04 | sean-k-mooney | we disabel os on windows for posix reasons | |
| 18:29:30 | sean-k-mooney | so i can push a patch and see what that does but i need to look at the tempest job | |
| 18:29:51 | sean-k-mooney | the unit/fucniton test failrues dont really tell us a lot about what the actual effect of the chagne is | |
| 18:30:56 | sean-k-mooney | ok devstack only deploy nova-api neutron failed to start so we didnt finish stacking | |
| 18:31:19 | sean-k-mooney | nova api seams to be up | |
| 18:31:54 | sean-k-mooney | neutron.plugins.ml2.ovo_rpc RuntimeError: cannot notify on un-acquired lock | |
| 18:32:02 | sean-k-mooney | that is the neutron server error | |
| 18:32:38 | sean-k-mooney | oh thats coming form oslo messaging | |
| 18:32:42 | sean-k-mooney | https://5ce7fa2fbddee5b97128-6f49bdea8e90fe6675f1ae011c4f0f22.ssl.cf5.rackcdn.com/933257/1/check/tempest-full-py3/018987a/controller/logs/screen-q-svc.txt | |
| 18:36:36 | sean-k-mooney | frickler: i think its better to phase this as the new eventlet release is breaking oslo adn that is breakign everything else | |
| 18:36:58 | sean-k-mooney | the nova unit/funtional test failure might eb related or flaky tests so i woudl try at least one recheck to confirm that | |
| 18:39:31 | sean-k-mooney | frickler: the fact the devstack functional job worked is also odd | |
| 18:40:03 | dansmith | JayF: sean-k-mooney: https://github.com/openstack/nova/blob/icehouse-eol/nova/cmd/__init__.py | |
| 18:40:05 | dansmith | os=False | |
| 18:40:39 | sean-k-mooney | rigth so we still do that for winwos | |
| 18:40:42 | sean-k-mooney | *windows | |
| 18:40:58 | dansmith | but that was for every binary on every platform, AFAIK | |
| 18:41:00 | JayF | I think that was required to be added some point around t? u? v? era-ish for greendns to behave properly? My memory is very lossy but I remember having to back os=False outta IPA at some point | |
| 18:41:03 | dansmith | ...at the time | |
| 18:41:20 | sean-k-mooney | it was yes | |
| 18:41:28 | sean-k-mooney | im trying to see when we changed it | |
| 18:41:43 | sean-k-mooney | we sould be able to go back but i am not sure its breaking nova | |
| 18:42:22 | dansmith | it was that way through rocky at least | |
| 18:42:42 | dansmith | I lose the thread when the refactor to monkey_patch.py happened | |
| 18:42:48 | dansmith | anyway, | |
| 18:43:05 | dansmith | I'm not saying we should, I'm defending why I thought the os.open thing wouldn't affect us :) | |
| 18:43:08 | dansmith | and proving that I'm old | |
| 18:43:22 | sean-k-mooney | dansmith: the sdk devstack tests passed https://5355e5989bc73779bf10-cc682349d9014dd293929a9a1be5247d.ssl.cf5.rackcdn.com/933257/1/check/openstacksdk-functional-devstack/fad2996/testr_results.html | |
| 18:46:20 | sean-k-mooney | dansmith: https://github.com/openstack/nova/commit/4f2d4d03f3a1a0a277dd04ec2abcbee2d4018fa2 | |
| 18:47:27 | sean-k-mooney | so there was no reason given why it orginally was false and also no reason given why we shoudl remov it | |
| 18:47:38 | dansmith | yeah :) | |
| 18:47:55 | sean-k-mooney | dansmith: by the way i also remember os=false for what its worth | |
| 18:48:24 | sean-k-mooney | as in that what i expected to see on master | |
| 18:48:52 | sean-k-mooney | for now i just recheck the requiremnt patch to see if things are failing the same way consitently | |
| 18:49:51 | sean-k-mooney | hum we changed in stein | |
| 18:58:26 | opendevreview | Ghanshyam proposed openstack/osc-placement master: Update python classifier in setup.cfg https://review.opendev.org/c/openstack/osc-placement/+/931113 | |
| 19:12:03 | sean-k-mooney | frickler: i was able to repoduce the nova.tests.unit.test_service.TestWSGIServer.test_server_pool_waitall failure locally | |
| 19:13:30 | sean-k-mooney | nova.tests.unit.test_service.TestWSGIService has two failing unit tests for me too because we are not stubing calsl to bind but that happened with our without the new eventlet | |
| 19:24:15 | frickler | sean-k-mooney: maybe the results also vary between python versions/distros. the cross jobs run with py311 on bookworm | |
| 19:28:33 | sean-k-mooney | im using debian testing on 3.12 | |
| 19:28:46 | sean-k-mooney | frickler: the unit test fialure is because its too specific | |
| 19:29:03 | sean-k-mooney | the unit test is ment to assert that waitall is called when we shutdown the wsgi server | |
| 19:29:17 | sean-k-mooney | but its asserting its called once and now its called twice | |
| 19:29:25 | sean-k-mooney | so the fix for that is just change the assert | |
| 19:29:39 | sean-k-mooney | im repoducing the functional test failure now | |
| 19:40:38 | sean-k-mooney | the functionatl test that is failing is restarting the scheduler and that reulst in self.service_ref being None | |
| 19:40:55 | sean-k-mooney | i think this is an artifact of how we are doing that | |
| 19:41:45 | sean-k-mooney | restarting services in our functional tests is kind of a hack | |
| 19:42:19 | sean-k-mooney | so again i think this will be a case of a slight test tweak but not an actual regression | |
| 19:47:37 | opendevreview | sean mooney proposed openstack/nova master: [eventlet] update nova tests for eventlet 0.37.0 https://review.opendev.org/c/openstack/nova/+/933365 | |
| 19:47:45 | sean-k-mooney | frickler: ^ i think that will do it | |
| 19:49:13 | sean-k-mooney | i dont think that matrially changes the intent of any of those tests so if that passes ci we can merge it and at least get those jobs passign with eventlet 0.37.0 | |
| 19:49:25 | sean-k-mooney | i can try installing that in a devstack tomorrow or next week | |
| 19:49:43 | sean-k-mooney | assumign the requirements patch does not pass tempest-py3 this time | |
| 19:52:31 | sean-k-mooney | ok im going to call it a day chat to folks tomorrow o/ | |
| 20:47:06 | gouthamr | bauzas: delayed response wrt your question regarding the virtiofs topic.. there's a conflict tomorrow at 1400 UTC tomorrow, and there are TC sessions beyond that.. I don't mind us chatting about this async / over email.. I mainly wanted to understand where it falls in the team's priorities and what we/anyone can do to help | |
| #openstack-nova - 2024-10-25 | |||
| 02:37:35 | opendevreview | Ghanshyam proposed openstack/nova master: Update gate jobs as per the 2025.1 cycle testing runtime https://review.opendev.org/c/openstack/nova/+/932648 | |
| 03:07:00 | opendevreview | Ghanshyam proposed openstack/nova master: Update gate jobs as per the 2025.1 cycle testing runtime https://review.opendev.org/c/openstack/nova/+/932648 | |
| 03:26:26 | opendevreview | Takashi Kajinami proposed openstack/nova master: Stop asking bug report to upstream https://review.opendev.org/c/openstack/nova/+/933381 | |
| 03:26:42 | tkajinam | ^^^ I wonder if can consider this switch now | |
| 05:04:59 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use native DictOpt for disk_cachemode https://review.opendev.org/c/openstack/nova/+/915390 | |
| 05:05:05 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Validate valid cache mode at config input layer https://review.opendev.org/c/openstack/nova/+/933327 | |
| 06:16:21 | bauzas | tkajinam: gibi: fwiw, the i18n team will have a session at 7am UTC related to the documentation translation, I'll be there | |
| 06:18:58 | bauzas | tkajinam: sure about the launchpad bug, but I have some concern | |
| 06:25:46 | tkajinam | bauzas, thx. I need to attend an internal meeting soon but if that finishes early then I'll join it | |
| 06:26:35 | tkajinam | we used to have number of people here in Japan who lead the translation work but most of these already left. that's a sad part of the discussion :-( | |
| 06:27:09 | tkajinam | bauzas, yeah there would be a tradeoffs but as we see more case where a reporter says " I was asked to report a bug so I did it" I think setting a better expectation might be helpful for both sides | |
| 06:31:24 | bauzas | tkajinam: yup indeed a tradeoff, I'll try to review your patch this morning my time | |
| 06:56:01 | bauzas | I wonder how many I'll have | |
| 07:00:22 | bauzas | people can chime in if they want | |
| 08:13:26 | gibi | bauzas: sorry but that was a late notice for me | |
| 08:13:36 | bauzas | gibi: no worries at all | |
| 08:14:13 | bauzas | given the weekdays we have, that's abolutely normal to take some break | |
| 08:14:46 | bauzas | one day we will have physical PTGs and that won't longer be a problem (half kidding) | |
| 08:16:18 | bauzas | anyway, the most important bit for nova to know is that translation patches to the doc would have a specific gerrit policy where language moderators would have a gerrit label | |
| 08:16:41 | bauzas | so a single core approval could be done based on the label | |
| 08:17:04 | bauzas | nothing harder than the current model we have with .pot files | |
| 08:19:05 | gibi | ack, thanks for the summary | |
| 08:53:08 | r-taketn | Hi, I want to use MIG(multi-instance GPU) on OpenStack VM by using NVIDIA A100. Must I install NVIDIA vGPU driver to use MIG on OpenStack VM? I cannot see /sys/class/mdev_bus, so I cannot use vGPU method . And I cannot see mig device by lspci so I cannot use pci-passthrough. | |
| 09:05:57 | pas-ha[m] | r-taketn: you must install latest NVIDIA AI Enterprise drivers on the compute node, and then switch the card to the MIG mode. Then partition it to MIGs. | |
| 09:07:09 | pas-ha[m] | the vGPU drivers no longer support A100 (for about a year already), everything that is MIG-capable requires installation of NVIDIA AI Enterprise | |
| 09:07:49 | pas-ha[m] | https://docs.nvidia.com/ai-enterprise/latest/user-guide/index.html | |
| 09:36:19 | r-taketn | pas-ha[m]: Thank you for reply. I read AI Enterprise driver manual. I only installed free nvidia-driver, but it looks like I could partition a GPU device to MIGs by using `nvidia-smi -i 1 -mig 1` and `nvidia-smi mig -i 0 -cgi 2g.20gb -C`. Is it just hardware-level partitioning? So must I install not-free AI Enterprise driver to use MIG on VMs or applications? | |
| 09:39:42 | pas-ha[m] | honestly not sure how would that work with 'free' driver. did you actually tried to perform those commands? the command may be there (nvidia-smi is installed even by the desktop GPU drivers), but does it do anything? anyway, you must switch the card to the SR-IOV mode first for it to work in the MIG mode `/usr/lib/nvidia/sriov-manage -e <PCI-ADDRESS-OF-NVIDIA-CARD>` | |
| 09:41:15 | pas-ha[m] | then you will get 16 extra PCI devices for this card, those are VFs for SR-IOV, you will create the MIGs on them and pass these VF PCI devices to the VMs. | |
| 09:52:10 | bauzas | r-taketn: sorrry was in a meeting | |
| 09:52:28 | bauzas | MIG-mode is somehow doable with Nova | |
| 09:52:42 | bauzas | but you need to pre-create the GIs | |
| 09:53:07 | bauzas | as pas-ha[m] said, you also need to pre-create the VFs | |
| 09:53:41 | bauzas | once you create the graphical instances, you should be able to use the related mdevs | |
| 09:55:22 | bauzas | please read https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#configuring-vgpu-manager-linux-with-kvm for how to enable VFs and GIs | |
| 09:56:08 | bauzas | for MIG, this is a bit later in the nvidia doc : https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#configuring-a-gpu-for-mig-backed-vgpus | |
| 10:00:52 | r-taketn | pas-ha[m], bauzas: I already created GIs. Sorry, I didn't understand mdev types and VFs. So I try to enable SR-IOV and create the VFs. Thanks. | |
| 10:32:29 | r-taketn | bauzas, pas-ha: I cannot find /usr/lib/nvidia/sriov-manage on my environment. https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#preparing-virtual-function-for-sr-iov-vgpu shows that Use only the custom script sriov-manage provided by NVIDIA AI Enterprise for the purpose. | |
| 10:33:25 | pas-ha[m] | yes, as I said, to properly use MIG you need AI Enterprise drivers... | |
| 10:35:47 | r-taketn | I got it. thank you for good advices | |