| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-10-25 | |||
| 03:26:42 | tkajinam | ^^^ I wonder if can consider this switch now | |
| 05:04:59 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Use native DictOpt for disk_cachemode https://review.opendev.org/c/openstack/nova/+/915390 | |
| 05:05:05 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Validate valid cache mode at config input layer https://review.opendev.org/c/openstack/nova/+/933327 | |
| 06:16:21 | bauzas | tkajinam: gibi: fwiw, the i18n team will have a session at 7am UTC related to the documentation translation, I'll be there | |
| 06:18:58 | bauzas | tkajinam: sure about the launchpad bug, but I have some concern | |
| 06:25:46 | tkajinam | bauzas, thx. I need to attend an internal meeting soon but if that finishes early then I'll join it | |
| 06:26:35 | tkajinam | we used to have number of people here in Japan who lead the translation work but most of these already left. that's a sad part of the discussion :-( | |
| 06:27:09 | tkajinam | bauzas, yeah there would be a tradeoffs but as we see more case where a reporter says " I was asked to report a bug so I did it" I think setting a better expectation might be helpful for both sides | |
| 06:31:24 | bauzas | tkajinam: yup indeed a tradeoff, I'll try to review your patch this morning my time | |
| 06:56:01 | bauzas | I wonder how many I'll have | |
| 07:00:22 | bauzas | people can chime in if they want | |
| 08:13:26 | gibi | bauzas: sorry but that was a late notice for me | |
| 08:13:36 | bauzas | gibi: no worries at all | |
| 08:14:13 | bauzas | given the weekdays we have, that's abolutely normal to take some break | |
| 08:14:46 | bauzas | one day we will have physical PTGs and that won't longer be a problem (half kidding) | |
| 08:16:18 | bauzas | anyway, the most important bit for nova to know is that translation patches to the doc would have a specific gerrit policy where language moderators would have a gerrit label | |
| 08:16:41 | bauzas | so a single core approval could be done based on the label | |
| 08:17:04 | bauzas | nothing harder than the current model we have with .pot files | |
| 08:19:05 | gibi | ack, thanks for the summary | |
| 08:53:08 | r-taketn | Hi, I want to use MIG(multi-instance GPU) on OpenStack VM by using NVIDIA A100. Must I install NVIDIA vGPU driver to use MIG on OpenStack VM? I cannot see /sys/class/mdev_bus, so I cannot use vGPU method . And I cannot see mig device by lspci so I cannot use pci-passthrough. | |
| 09:05:57 | pas-ha[m] | r-taketn: you must install latest NVIDIA AI Enterprise drivers on the compute node, and then switch the card to the MIG mode. Then partition it to MIGs. | |
| 09:07:09 | pas-ha[m] | the vGPU drivers no longer support A100 (for about a year already), everything that is MIG-capable requires installation of NVIDIA AI Enterprise | |
| 09:07:49 | pas-ha[m] | https://docs.nvidia.com/ai-enterprise/latest/user-guide/index.html | |
| 09:36:19 | r-taketn | pas-ha[m]: Thank you for reply. I read AI Enterprise driver manual. I only installed free nvidia-driver, but it looks like I could partition a GPU device to MIGs by using `nvidia-smi -i 1 -mig 1` and `nvidia-smi mig -i 0 -cgi 2g.20gb -C`. Is it just hardware-level partitioning? So must I install not-free AI Enterprise driver to use MIG on VMs or applications? | |
| 09:39:42 | pas-ha[m] | honestly not sure how would that work with 'free' driver. did you actually tried to perform those commands? the command may be there (nvidia-smi is installed even by the desktop GPU drivers), but does it do anything? anyway, you must switch the card to the SR-IOV mode first for it to work in the MIG mode `/usr/lib/nvidia/sriov-manage -e <PCI-ADDRESS-OF-NVIDIA-CARD>` | |
| 09:41:15 | pas-ha[m] | then you will get 16 extra PCI devices for this card, those are VFs for SR-IOV, you will create the MIGs on them and pass these VF PCI devices to the VMs. | |
| 09:52:10 | bauzas | r-taketn: sorrry was in a meeting | |
| 09:52:28 | bauzas | MIG-mode is somehow doable with Nova | |
| 09:52:42 | bauzas | but you need to pre-create the GIs | |
| 09:53:07 | bauzas | as pas-ha[m] said, you also need to pre-create the VFs | |
| 09:53:41 | bauzas | once you create the graphical instances, you should be able to use the related mdevs | |
| 09:55:22 | bauzas | please read https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#configuring-vgpu-manager-linux-with-kvm for how to enable VFs and GIs | |
| 09:56:08 | bauzas | for MIG, this is a bit later in the nvidia doc : https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#configuring-a-gpu-for-mig-backed-vgpus | |
| 10:00:52 | r-taketn | pas-ha[m], bauzas: I already created GIs. Sorry, I didn't understand mdev types and VFs. So I try to enable SR-IOV and create the VFs. Thanks. | |
| 10:32:29 | r-taketn | bauzas, pas-ha: I cannot find /usr/lib/nvidia/sriov-manage on my environment. https://docs.nvidia.com/ai-enterprise/5.1/user-guide/index.html#preparing-virtual-function-for-sr-iov-vgpu shows that Use only the custom script sriov-manage provided by NVIDIA AI Enterprise for the purpose. | |
| 10:33:25 | pas-ha[m] | yes, as I said, to properly use MIG you need AI Enterprise drivers... | |
| 10:35:47 | r-taketn | I got it. thank you for good advices | |
| 10:38:28 | frickler | sean-k-mooney: tested your eventlet fix and now it seems to hang somewhere, but I'm not sure which test is actually getting stuck https://zuul.opendev.org/t/openstack/build/dde58528ed9841f4b21d0091750fa176 | |
| 10:39:02 | frickler | at least the functional job is passing now | |
| 10:39:15 | sean-k-mooney | so thats the unit test job | |
| 10:39:27 | sean-k-mooney | and i only changed form assert_called_once to assert_called | |
| 10:39:42 | sean-k-mooney | so that not related to the change its somethign deeper | |
| 10:40:54 | sean-k-mooney | frickler: lookign at the logs the tests were still running and passing form what i can see | |
| 10:40:58 | sean-k-mooney | and we just hit the timeout | |
| 10:43:00 | sean-k-mooney | frickler: im not seing anything hang in the logs or even tak a large amoutn of time | |
| 10:46:31 | sean-k-mooney | it took about 3 mins to get to the poitn of runnign tox which does not seem exceeive 4.5 total to run the first test | |
| 10:47:20 | sean-k-mooney | oh i see | |
| 10:47:22 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/dde58528ed9841f4b21d0091750fa176/log/job-output.txt#33835-33836 | |
| 10:47:36 | sean-k-mooney | the absence of logs it just stopped making progress | |
| 10:48:15 | sean-k-mooney | i could just turn off os monkey patching as well | |
| 10:48:57 | sean-k-mooney | what im most concerned about by the way is this breaking oru syncronise decorator | |
| 10:49:06 | sean-k-mooney | or something like that | |
| 12:20:37 | sean-k-mooney | bauzas: so last firday, contractors were ment to come and do a solar site survay, they called in sick and it was reschduled for the top of the hour today. so i might miss part of the start of the ptg sessions | |
| 12:26:25 | bauzas | sean-k-mooney: ack np, we're already mostly done with the agenda :) | |
| 12:26:46 | sean-k-mooney | i have already left most of my feedback anyway | |
| 12:26:58 | sean-k-mooney | but i was actully expecting use to get throught he adgenda yesterday | |
| 12:27:15 | sean-k-mooney | hopefully that mean we can wrap up nova before the tc sessions | |
| 12:27:30 | bauzas | yup :) | |
| 12:27:42 | bauzas | I also need to attend the TC sessions :) | |
| 12:28:07 | sean-k-mooney | i do not need too but it would be nice ot have the option | |
| 13:01:44 | bauzas | stephenfin: do you want to join us in the PTG for your topic ? | |
| 13:01:56 | bauzas | we will discuss it around the next hour | |
| 13:29:11 | MengyangZhang[m] | <tkajinam> "you can put your topic at the..." <- so my topic was already discussed yesterday? I thought it would be discussed on Friday. | |
| 13:30:14 | tkajinam | MengyangZhang[m], we had cross project session between nova and cinder and that topic was raised during that discussion. nova team asked cinder team to double check if the proposal is valid or not from cinder's PoV | |
| 13:30:32 | tkajinam | MengyangZhang[m], if you have anything else you want to specifically discuss then you can re-add it to the topic for today, probably | |
| 13:30:34 | tkajinam | bauzas, ^^^ | |
| 13:31:01 | tkajinam | sorry I'm now in oceanbase discussion. I didn't plan it but noone form oslo was there | |
| 13:32:54 | MengyangZhang[m] | tkajinam: then I guess we are all good. If you need anything from me, please let me know | |
| 13:35:48 | MengyangZhang[m] | MengyangZhang[m]: I also have the implementation ready if it helps the community understand the scope of the problem. | |
| 14:02:12 | sean-k-mooney | MengyangZhang[m]: the main opens we have is jsut are all the qos polices you want to enabel already valid and part of cinder | |
| 14:02:49 | sean-k-mooney | MengyangZhang[m]: nova cant enabel any front end qos for cinder voluems unless the cinder team has already enabled support for it in there api | |
| 14:03:17 | sean-k-mooney | if its not supported on the cinder side today you need to first enabel it there and then in nova | |
| 14:05:45 | tkajinam | re-reviewing the spec it says that we register these specs in cinder but actually applies the limit in nova. I don't think that's something already done | |
| 14:05:59 | tkajinam | we have quota:disk_write_bytes_sec but it's set for nova flavor | |
| 14:09:47 | tkajinam | I probably have to check if cinder exposes any qos elements in connection info | |
| 14:13:14 | artom | Coming late because of baby's doctor appointment - is PTG finished? | |
| 14:13:37 | artom | Sounds like we ended with the OpenAPI topic | |
| 14:19:14 | tkajinam | so I reminded bauzas of taking a team photo this afternoon but missed the chance to be part of it X-( | |
| 14:19:19 | tkajinam | (hoping he took one | |
| 14:34:58 | gibi | tkajinam: at least he did not take it openly:) | |
| 14:35:18 | gibi | artom: yepp we finished with the nova agenda. There is two tc sessions left | |
| 14:55:42 | stephenfin | bauzas: sean-k-mooney: fyi, the spec has already been re-proposed and merged :) https://review.opendev.org/c/openstack/nova-specs/+/928800 | |
| 14:56:19 | stephenfin | sorry, I wasn't checking IRC :( | |
| 16:50:17 | opendevreview | Dan Smith proposed openstack/nova master: DNM: Test with latest oslo.utils https://review.opendev.org/c/openstack/nova/+/933444 | |
| 17:01:50 | tkajinam | long day, long week | |
| 17:02:59 | sean-k-mooney | yep | |
| 17:03:04 | sean-k-mooney | but productive i think | |
| 17:03:12 | tkajinam | yeah | |
| 17:03:22 | sean-k-mooney | less so then in person but still we used the time well i think | |
| 17:04:09 | tkajinam | yeah I think we have adopted a lot to the current vPTG style | |
| 17:04:12 | gibi | I agree, we had good discussions with actionable outcomes | |
| 17:04:30 | sean-k-mooney | one of my friends remined me its a public holiday in ireland on monday. so im looking forward to the long weekend | |
| 17:04:51 | sean-k-mooney | gibi: by the wya im hoping ti finish reviewing the igb series before i finish today | |
| 17:05:43 | gibi | sean-k-mooney: if not, it is not a problem at all | |
| 17:06:01 | sean-k-mooney | im on the second last change so that and the docs one | |
| 17:06:05 | gibi | :) | |
| 17:06:06 | sean-k-mooney | sofar looks good to me | |
| 17:07:03 | tkajinam | leaving now. have a good weekend | |
| 17:07:23 | gibi | tkajinam: same to you | |
| #openstack-nova - 2024-10-26 | |||
| 00:41:08 | opendevreview | sean mooney proposed openstack/nova master: [WIP][scheduler] add hash based weigher https://review.opendev.org/c/openstack/nova/+/933478 | |