| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-01-17 | |||
| 18:59:40 | bauzas | then, look at the available_instances value for the type | |
| 18:59:45 | bauzas | for each of the VFs | |
| 19:00:12 | bauzas | then, please look at how we create the inventories : | |
| 19:00:49 | bauzas | https://github.com/openstack/nova/blob/a72f7eaac78927892b937d451cbacb24a83c05ac/nova/virt/libvirt/driver.py#L8128-L8171 | |
| 19:01:07 | bauzas | we count the number of mdevs that are already created | |
| 19:01:16 | _colby | ok thanks! Appreciate the help. Will dig some more into this today. | |
| 19:01:36 | bauzas | and then we add it with the value of available_instances for the VF related type | |
| 19:02:10 | bauzas | once you create all the mdevs, you should have VGPU inventories only for the VFs that have mdevs | |
| 19:02:32 | bauzas | and then you should delete the other RPs related the VFs that have none of mdevs | |
| 19:02:50 | bauzas | (unfortunately, due to the bug report, we don't remove automatically them, but my series will do it) | |
| 19:04:03 | bauzas | _colby: I'm in the CET timezone but you can in general ping me until 6pm UTC | |
| 23:48:23 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | |
| #openstack-nova - 2024-01-18 | |||
| 00:09:24 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | |
| 02:03:02 | opendevreview | melanie witt proposed openstack/nova master: DNM test ephemeral encryption + resize: qcow2, raw, rbd https://review.opendev.org/c/openstack/nova/+/862416 | |
| 02:10:16 | opendevreview | likui proposed openstack/nova master: Add py3.11 in tox file https://review.opendev.org/c/openstack/nova/+/901587 | |
| 02:10:42 | opendevreview | likui proposed openstack/nova master: Update python classifier in setup.cfg https://review.opendev.org/c/openstack/nova/+/901587 | |
| 02:11:55 | tonyb | Any chance I can get some eyes on: https://review.opendev.org/c/openstack/nova/+/905314 ? | |
| 06:50:39 | opendevreview | Amit Uniyal proposed openstack/nova master: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/904817 | |
| 08:28:06 | bauzas | tonyb: sent to the gate | |
| 09:21:39 | opendevreview | Merged openstack/nova master: Fix test failures with oslo.limit 2.3.0 https://review.opendev.org/c/openstack/nova/+/905314 | |
| 09:23:53 | tonyb | bauzas: thank you | |
| 09:24:20 | tonyb | melwitt: and thank you too for the quick review | |
| 14:37:35 | opendevreview | Merged openstack/nova master: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/905684 | |
| 15:16:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: Add a functest for verifying multiple VGPU allocations https://review.opendev.org/c/openstack/nova/+/845747 | |
| 15:16:37 | opendevreview | Sylvain Bauza proposed openstack/nova master: Support multiple allocations for vGPUs https://review.opendev.org/c/openstack/nova/+/845757 | |
| 16:40:37 | opendevreview | Merged openstack/nova master: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/904817 | |
| 16:53:35 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:16:43 | _colby | bauzas: thanks for your help yesterday. So from what you said do we need to manually create the mdevs now? I was confused on that part. I see all the VF and available_instances for them all. But mdevctl shows no devices and nova does not seem to create them any more. So is manual creation required now? | |
| 17:17:05 | bauzas | (on a meeting now but I'll reply later) | |
| 17:17:10 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:30:41 | bauzas | _colby: I'll need to disappear shortly, but please explain me more : are you able to see mediated devices in /sys/bus/mdev ? | |
| 17:32:52 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:38:47 | _colby | bauzas: yes I see 1 here for all the available VFs: cat /sys/class/mdev_bus/*/mdev_supported_types/nvidia-563/available_instances | |
| 17:39:05 | bauzas | so basically all GPU VFs are available | |
| 17:39:13 | bauzas | do you have instances on the host ? | |
| 17:39:25 | _colby | no this is a fresh host Im testing with | |
| 17:39:31 | bauzas | and do you have placement allocations ? | |
| 17:40:50 | bauzas | _colby: re: the bug I mentioned yesterday, once you create enough mdevs equal to the max number of vGPUs the type can create, then all other VFs will start having 0 available instances but placement will continue to say 'sure, you can use them' | |
| 17:41:13 | _colby | yes the resource provider shows all the VF addresses as seperate items and looking at inventory of one I see the expected output: VGPU, allocation_ratio=1.0,min_unit=1,max_unit=1,reserved=0,step_size=1,total=1,used=0 | |
| 17:41:14 | bauzas | that's why I'm recommending to precreate the mediated devices on the VFs you want and then start nova-compute | |
| 17:41:33 | bauzas | oh, you already started nova-computz | |
| 17:41:35 | bauzas | then, | |
| 17:41:46 | bauzas | 1/ create the mdevs by hand (or by mdevctl) | |
| 17:41:53 | bauzas | 2/ stop nova-compute | |
| 17:42:05 | bauzas | 3/ delete the resource providers matching the unused VFs | |
| 17:42:10 | bauzas | 4/ restart nova-compute | |
| 17:42:27 | bauzas | you should then have placement reflecting the correct resources | |
| 17:42:45 | bauzas | I need to leave now | |
| 17:43:08 | _colby | ok I can do that. Seems like a bit of heavy admin work every time we want to reconfig the vGPU slices. Nova does not want to create them automatically any more? | |
| 17:43:55 | bauzas | that's the bug report I mentioned yesterday | |
| 17:44:05 | _colby | but if that will get it working then we can do that | |
| 17:44:14 | bauzas | nova can continue to create the mediated devices | |
| 17:45:08 | bauzas | but as soon as you create the Nth mdev from a GPU (where N equals the max number of vGPUs you can create from a type), then you'll have inconsistencies between placement and sysfs | |
| 17:45:22 | _colby | I see | |
| 17:45:33 | _colby | but in our case Nova does not seem to be creating any mdevs | |
| 17:45:33 | bauzas | a fix is there, we gonna review it this cycle | |
| 17:45:51 | bauzas | I need to go but shouldn't be | |
| 17:46:07 | _colby | no problem thanks | |
| 17:46:29 | bauzas | I can't remember when we added more logs to _allocate_mdevs() but yoga seems recent enough to have those | |
| 17:48:32 | bauzas | https://github.com/openstack/nova/commit/6feb3350b048606297068841e3feba110bb0b0ab | |
| 17:48:46 | bauzas | nah, antelope | |
| 17:49:41 | _colby | I can manually patch our system to add these logs for testing | |
| 18:19:47 | _colby | bauzas: whenever you get back I made more progress: https://pastebin.com/Xv2bvaf7 | |
| 18:20:24 | _colby | its compaining about no available instances on that device but there is: cat /sys/class/mdev_bus/0000\:17\:02.0/mdev_supported_types/nvidia-563/available_instances | |
| 18:22:22 | _colby | also that device is not listed in the config for that type: device_addresses=0000:17:00.4,0000:17:00.5,0000:17:00.6,0000:17:00.7 | |
| 21:51:51 | opendevreview | Merged openstack/nova master: Allow config to support virtiofs (driver) https://review.opendev.org/c/openstack/nova/+/886522 | |
| 21:51:59 | opendevreview | Merged openstack/nova master: Attach Manila shares via virtiofs (db) https://review.opendev.org/c/openstack/nova/+/831193 | |
| #openstack-nova - 2024-01-19 | |||
| 00:36:34 | opendevreview | Takashi Natsume proposed openstack/nova-specs master: Create specs directory for 2024.2 Dalmatian https://review.opendev.org/c/openstack/nova-specs/+/906073 | |
| 05:23:47 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/906088 | |
| 05:23:48 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/906089 | |
| 05:44:49 | opendevreview | Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648 | |
| 05:44:49 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 05:44:50 | opendevreview | Amit Uniyal proposed openstack/nova master: Removed explicit call to delete attachment https://review.opendev.org/c/openstack/nova/+/891289 | |
| 08:32:35 | bauzas | _colby: I'm here now but let's discuss this when you're back, I have some questions (saw your pastebin) | |
| 09:19:05 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/906089 | |
| 09:55:03 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Configure and teardown ephemeral encryption secrets https://review.opendev.org/c/openstack/nova/+/826754 | |
| 09:55:04 | opendevreview | melanie witt proposed openstack/nova master: imagebackend: Add support to libvirt_info for LUKS based encryption https://review.opendev.org/c/openstack/nova/+/826755 | |
| 09:55:04 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to convert_image https://review.opendev.org/c/openstack/nova/+/870934 | |
| 09:55:05 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 09:55:05 | opendevreview | melanie witt proposed openstack/nova master: Support resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933 | |
| 09:55:06 | opendevreview | melanie witt proposed openstack/nova master: Add hw_ephemeral_encryption_secret_uuid image property https://review.opendev.org/c/openstack/nova/+/870935 | |
| 09:55:06 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | |
| 09:55:07 | opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870937 | |
| 09:55:07 | opendevreview | melanie witt proposed openstack/nova master: Support rebuild and unshelve with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870939 | |
| 09:55:09 | opendevreview | melanie witt proposed openstack/nova master: Support rescue with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/873675 | |
| 09:55:09 | opendevreview | melanie witt proposed openstack/nova master: libvirt: make |
|
| 09:55:11 | opendevreview | melanie witt proposed openstack/nova master: WIP Support migration with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/905512 | |
| 09:55:11 | opendevreview | melanie witt proposed openstack/nova master: Reject (resize|rebuild) API requests with conflicting encryption https://review.opendev.org/c/openstack/nova/+/904240 | |
| 09:55:13 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | |
| 09:55:13 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | |
| 09:55:15 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 10:46:02 | opendevreview | Sylvain Bauza proposed openstack/nova master: libvirt: Cap with max_instances GPU types https://review.opendev.org/c/openstack/nova/+/899625 | |
| 10:46:02 | opendevreview | Sylvain Bauza proposed openstack/nova master: vgpu: Allow device_addresses to not be set https://review.opendev.org/c/openstack/nova/+/902084 | |
| 10:46:03 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP: add coverage for registered dyn opts https://review.opendev.org/c/openstack/nova/+/902771 | |
| 10:54:54 | opendevreview | Sylvain Bauza proposed openstack/nova master: FUP: add coverage for registered dyn opts https://review.opendev.org/c/openstack/nova/+/902771 | |
| 13:18:46 | opendevreview | Tobias Urdin proposed openstack/nova master: Fail nova-compute startup when RP cannot be created https://review.opendev.org/c/openstack/nova/+/904381 | |
| 13:25:04 | opendevreview | Tobias Urdin proposed openstack/nova master: libvirt: set remaining to 0 when no disk to migrate https://review.opendev.org/c/openstack/nova/+/873846 | |
| 13:25:09 | opendevreview | Tobias Urdin proposed openstack/nova master: libvirt: set remaining to 0 when no disk to migrate https://review.opendev.org/c/openstack/nova/+/873846 | |
| 13:29:28 | opendevreview | Tobias Urdin proposed openstack/nova master: Fix wrong nova-manage command in upgrade check https://review.opendev.org/c/openstack/nova/+/880819 | |
| 13:33:50 | opendevreview | Tobias Urdin proposed openstack/nova master: Remove libvirt tunnelled migration https://review.opendev.org/c/openstack/nova/+/879021 | |