| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-06-23 | |||
| 05:28:12 | frickler | ade_lee: yes, that's the same failure pattern that caused us to make c9s non-voting in devstack | |
| 08:27:14 | bauzas | gibi: others, fwiw, I'll be on PTO tomorrow | |
| 08:27:24 | gibi | bauzas: o/ ack | |
| 09:15:51 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds validation for hw machine type in host caps https://review.opendev.org/c/openstack/nova/+/847126 | |
| 10:06:59 | sean-k-mooney | ade_lee: assuming this is related to fips and centos 9 stream ya interface attach is broken due to an upstream libvirt bug which has a fix internally but is not relased yet | |
| 11:06:03 | opendevreview | Merged openstack/nova master: Change TooOldComputeService upgrade check code to failure https://review.opendev.org/c/openstack/nova/+/845262 | |
| 14:08:12 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds validation for hw machine type in host caps https://review.opendev.org/c/openstack/nova/+/847126 | |
| 14:31:20 | amorin | hello nova! | |
| 14:34:55 | amorin | I was looking for an idea on how to schedule instances on computes based on some metrics that are not cpu related | |
| 14:35:04 | amorin | like, the kernel version, the operating system version | |
| 14:40:35 | amorin | i was thinking that I could tweak the compute monitors (that collect metrics) | |
| 14:40:40 | amorin | but I dont know if this is the best idea? | |
| 14:45:05 | mnaser | amorin: maybe you could use traits out of bands? | |
| 14:45:21 | mnaser | like have something that ships the traits of those boxes to placement | |
| 14:45:23 | amorin | out of bands? | |
| 14:45:32 | amorin | like a custom traits | |
| 14:45:44 | mnaser | yeah i meant out of band as in you wont be in the nova code path to determine this | |
| 14:45:46 | amorin | yes, I did try that, but it seems not available in the nova weighter | |
| 14:45:59 | mnaser | ah so you need this info in the weigher and not in the filter stage | |
| 14:46:20 | amorin | yes, because I dont want to filter out the hosts | |
| 14:46:28 | amorin | I just want them to be lower in the priority | |
| 14:47:30 | mnaser | ah okay so my suggestion wont help then | |
| 14:47:42 | amorin | unfortunately :( but thanks! | |
| 14:47:44 | gibi | placement can only be used today to do filtering decision, there is no weighing support there | |
| 14:47:54 | gibi | so you need a now weigher code for that | |
| 14:48:01 | gibi | so you need nova weigher code for that | |
| 14:48:02 | amorin | yup, and doing placement call from weigher seems a bad idea | |
| 14:50:46 | gibi | I dont think we have code for that in nova yet, you can try to abuse the MetricsWeigher | |
| 14:50:53 | gibi | but it feels hackish | |
| 14:51:50 | amorin | thats exactly what I am going to do, but I just wanted to know if something else, less hacky would exist | |
| 14:52:41 | gibi | I'm not aware of any less hackish way now | |
| 14:55:13 | amorin | ok, thanks gibi ! | |
| 15:00:36 | bauzas | amorin: I was trying to reply to you about your question :) | |
| 15:01:02 | bauzas | yeah, you should use weights | |
| 15:01:10 | bauzas | placement doesn't support them | |
| 15:10:21 | amorin | bauzas yeah! thanks | |
| 15:10:32 | amorin | you can still answer, maybe that would be nice for other people :) | |
| 15:10:44 | bauzas | amorin: sorry, I didn't have time to reply | |
| 15:10:59 | bauzas | amorin: I'll try, but I'm off tomorrow :( | |
| 15:11:08 | amorin | no worries :) | |
| 17:49:39 | sean-k-mooney | bauzas: i will be off tomorrow to recover but if you can take a look at https://review.opendev.org/c/openstack/nova/+/847001 i think i adress all your issues | |
| #openstack-nova - 2022-06-24 | |||
| 02:29:30 | opendevreview | Merged openstack/python-novaclient master: Imported Translations from Zanata https://review.opendev.org/c/openstack/python-novaclient/+/846880 | |
| 08:38:31 | opendevreview | Takashi Natsume proposed openstack/python-novaclient master: Replace old URLs with new ones https://review.opendev.org/c/openstack/python-novaclient/+/840693 | |
| 09:50:53 | opendevreview | Stephen Finucane proposed openstack/os-resource-classes master: Update Python testing as oer zed cycle testing runtime https://review.opendev.org/c/openstack/os-resource-classes/+/847562 | |
| 09:51:11 | stephenfin | gibi: Easy review there ^ | |
| 09:57:24 | stephenfin | never mind, we had similar patches already that I hadn't spotted | |
| 10:00:26 | opendevreview | Merged openstack/os-resource-classes master: Update CI to use unversioned jobs template https://review.opendev.org/c/openstack/os-resource-classes/+/834537 | |
| 10:02:05 | opendevreview | Merged openstack/os-resource-classes master: setup: Update Python testing classifiers https://review.opendev.org/c/openstack/os-resource-classes/+/834643 | |
| 10:02:30 | opendevreview | Merged openstack/os-resource-classes master: setup: Replace dashes with underscores, add links https://review.opendev.org/c/openstack/os-resource-classes/+/834644 | |
| 10:42:54 | gibi | stephenfin: :) | |
| 11:11:22 | opendevreview | Merged openstack/placement master: Change minversion of tox to 3.18.0 https://review.opendev.org/c/openstack/placement/+/791971 | |
| 11:35:13 | opendevreview | Merged openstack/python-novaclient master: Replace old URLs with new ones https://review.opendev.org/c/openstack/python-novaclient/+/840693 | |
| 11:51:01 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds validation for hw machine type in host caps https://review.opendev.org/c/openstack/nova/+/847126 | |
| 13:47:42 | opendevreview | Takashi Natsume proposed openstack/os-resource-classes master: Change minversion of tox to 3.18.0 https://review.opendev.org/c/openstack/os-resource-classes/+/791974 | |
| 14:03:54 | opendevreview | ribaudr proposed openstack/nova master: Allow unshelve to a specific host (Compute API part) https://review.opendev.org/c/openstack/nova/+/831507 | |
| 14:03:55 | opendevreview | ribaudr proposed openstack/nova master: Allow unshelve to a specific host (REST API part) https://review.opendev.org/c/openstack/nova/+/845897 | |
| 14:26:55 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds validation for hw machine type in host caps https://review.opendev.org/c/openstack/nova/+/847126 | |
| 14:44:59 | opendevreview | ribaudr proposed openstack/nova master: Allow unshelve to a specific host (REST API part) https://review.opendev.org/c/openstack/nova/+/845897 | |
| 14:44:59 | opendevreview | ribaudr proposed openstack/nova master: Allow unshelve to a specific host (Compute API part) https://review.opendev.org/c/openstack/nova/+/831507 | |
| 17:49:43 | colby_ | Hello everyone. we are experienceing a strange but with vGPU. We are using Virtoria on Centos 8 Stream with A40 gpus in SRIOV setup. Everything works correctly except when we delete an instance. It does not seem to release the virtualfunction mdev device. I can manually release them by echoing to the remove method of the mdev. So its like nova is not doing that. | |
| 17:50:49 | colby_ | Are there any bugs about this? Can someone point me to the code that handles this so I can see if I can tell why its failing. Thanks. | |
| #openstack-nova - 2022-06-25 | |||
| 09:40:18 | frickler | https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013410 looks like a unit test that fails under load. expected: timed out after 0.00 seconds, actual: timed out after 0.02 seconds | |
| 15:09:52 | opendevreview | Artom Lifshitz proposed openstack/nova master: Broken WIP: Update libvirt enlightenments for Windows https://review.opendev.org/c/openstack/nova/+/847641 | |
| #openstack-nova - 2022-06-26 | |||
| 15:27:44 | opendevreview | Takashi Kajinami proposed openstack/osc-placement master: Replace deprecated assertRaisesRegexp https://review.opendev.org/c/openstack/osc-placement/+/847689 | |
| 15:28:44 | opendevreview | Takashi Kajinami proposed openstack/osc-placement master: Replace deprecated assertRaisesRegexp https://review.opendev.org/c/openstack/osc-placement/+/847689 | |
| 15:43:15 | tobias-urdin | colby_: iirc the mdev is the actual resource so it'll be reused | |
| 17:46:39 | colby_ | tobias-urdin: It does not seem to be the case. Nova gives the error that there are no resources available. If I manually remove the mdev then we can create more instances | |
| 17:57:01 | tobias-urdin | colby_: do you get any errors or similar? do you have some more info about what you are running, the only issue we have currently with vgpu part is https://review.opendev.org/c/openstack/nova/+/838976 | |
| 18:03:34 | colby_ | tobias-urdin: unfortunately no errors other than no resources. I have looked all through system logs for any sort of errors. mdevctl shows all the devices still used and nvidia-smi vgpu -c show no vgpu available. | |
| 18:04:54 | colby_ | if I do the following command: echo "1" > nvidia-563/devices/8f5be0ec-3a4b-4e72-b613-4e93a27951a0/remove | |
| 18:05:24 | colby_ | then nvidia-smi -c shows available gpu and mdevctl shows it not being used any more. I am then able to spin up another vgpu instance | |
| 18:06:06 | colby_ | that is being done in the /sys/bus/pci/devices/<virtual function cpi address>/mdev_supported_types directory | |
| 18:06:32 | colby_ | we are on Centos8 stream and using Victoria version of openstack currently | |
| 18:07:42 | colby_ | example mdevctl output: 67e63f1e-07f2-474f-874c-826a024c10ec 0000:21:01.7 nvidia-563 manual | |
| #openstack-nova - 2022-06-27 | |||
| 07:15:57 | bauzas | good morning Nova | |
| 07:24:34 | gibi | o/ | |
| 07:35:34 | bauzas | one day off and a lot of emails to triager | |
| 08:18:18 | gibi | frickler: re https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013410 we have an upstream bug for that https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1013410 and it is being fixed by https://review.opendev.org/c/openstack/nova/+/845922 | |
| 08:18:32 | gibi | bah, I mean upstream bug https://bugs.launchpad.net/nova/+bug/1978817 | |
| 08:26:36 | gibi | colby_: I think we saw similar issues before but I don't find the relevan bug report now I suggest to ping bauzas as he might have more context with vgpus/mdevs | |
| 08:26:55 | bauzas | context ? | |
| 08:27:35 | gibi | bauzas: 19:49 < colby_> Hello everyone. we are experienceing a strange but with vGPU. We are using Virtoria on Centos 8 Stream with A40 gpus in SRIOV setup. Everything works correctly except when we delete an instance. It does not seem to release the virtualfunction mdev device. I can manually release them by echoing to the remove method of the mdev. So its like nova is not doing that. | |
| 08:27:55 | gibi | bauzas: that was from friday | |
| 08:30:35 | frickler | gibi: ah, cool, thx for the pointer | |
| 08:30:47 | gibi | frickler: I replied in the debian tracker too | |
| 08:32:39 | bauzas | gibi: colby_: yes we don't delete the mdevs | |
| 08:33:09 | bauzas | 4 years ago, I thought it was not needed | |
| 09:30:47 | Uggla | Hello, bauzas, gibi, sean-k-mooney https://review.opendev.org/c/openstack/nova/+831507, I have fixed latest comments from bauzas and I have verified it works fine with devstack. Can you go ahead with a new review round ? | |
| 09:31:07 | sean-k-mooney | gibi: we are tarcking that as a downstream bug | |
| 09:31:28 | sean-k-mooney | im suggesting that we shoudl free the mdev when we delete the vm | |
| 09:31:39 | sean-k-mooney | we could resue the mdev on the next vm creation but | |
| 09:32:03 | sean-k-mooney | it think that will be problematic if we start using mdevs more dynamicaly in the future | |
| 09:32:20 | sean-k-mooney | it would work for our current usecase but i would prefer to only allocate mdevs if they are in use | |
| 09:33:02 | sean-k-mooney | Uggla: gerrit says no | |
| 09:33:18 | sean-k-mooney | Uggla: specifically it give a 404 | |
| 09:33:27 | sean-k-mooney | and a | |
| 09:33:29 | sean-k-mooney | ¯\_(ツ)_/¯ | |
| 09:33:45 | Uggla | https://review.opendev.org/c/openstack/nova-specs/+/833669 | |
| 09:34:11 | sean-k-mooney | that works fine | |
| 09:35:13 | Uggla | Here is the good one: https://review.opendev.org/c/openstack/nova/+/831507 | |