| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-01-18 | |||
| 16:53:35 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:16:43 | _colby | bauzas: thanks for your help yesterday. So from what you said do we need to manually create the mdevs now? I was confused on that part. I see all the VF and available_instances for them all. But mdevctl shows no devices and nova does not seem to create them any more. So is manual creation required now? | |
| 17:17:05 | bauzas | (on a meeting now but I'll reply later) | |
| 17:17:10 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:30:41 | bauzas | _colby: I'll need to disappear shortly, but please explain me more : are you able to see mediated devices in /sys/bus/mdev ? | |
| 17:32:52 | opendevreview | Doug Szumski proposed openstack/nova master: Use dedicated live migration network during pre-migration https://review.opendev.org/c/openstack/nova/+/906053 | |
| 17:38:47 | _colby | bauzas: yes I see 1 here for all the available VFs: cat /sys/class/mdev_bus/*/mdev_supported_types/nvidia-563/available_instances | |
| 17:39:05 | bauzas | so basically all GPU VFs are available | |
| 17:39:13 | bauzas | do you have instances on the host ? | |
| 17:39:25 | _colby | no this is a fresh host Im testing with | |
| 17:39:31 | bauzas | and do you have placement allocations ? | |
| 17:40:50 | bauzas | _colby: re: the bug I mentioned yesterday, once you create enough mdevs equal to the max number of vGPUs the type can create, then all other VFs will start having 0 available instances but placement will continue to say 'sure, you can use them' | |
| 17:41:13 | _colby | yes the resource provider shows all the VF addresses as seperate items and looking at inventory of one I see the expected output: VGPU, allocation_ratio=1.0,min_unit=1,max_unit=1,reserved=0,step_size=1,total=1,used=0 | |
| 17:41:14 | bauzas | that's why I'm recommending to precreate the mediated devices on the VFs you want and then start nova-compute | |
| 17:41:33 | bauzas | oh, you already started nova-computz | |
| 17:41:35 | bauzas | then, | |
| 17:41:46 | bauzas | 1/ create the mdevs by hand (or by mdevctl) | |
| 17:41:53 | bauzas | 2/ stop nova-compute | |
| 17:42:05 | bauzas | 3/ delete the resource providers matching the unused VFs | |
| 17:42:10 | bauzas | 4/ restart nova-compute | |
| 17:42:27 | bauzas | you should then have placement reflecting the correct resources | |
| 17:42:45 | bauzas | I need to leave now | |
| 17:43:08 | _colby | ok I can do that. Seems like a bit of heavy admin work every time we want to reconfig the vGPU slices. Nova does not want to create them automatically any more? | |
| 17:43:55 | bauzas | that's the bug report I mentioned yesterday | |
| 17:44:05 | _colby | but if that will get it working then we can do that | |
| 17:44:14 | bauzas | nova can continue to create the mediated devices | |
| 17:45:08 | bauzas | but as soon as you create the Nth mdev from a GPU (where N equals the max number of vGPUs you can create from a type), then you'll have inconsistencies between placement and sysfs | |
| 17:45:22 | _colby | I see | |
| 17:45:33 | bauzas | a fix is there, we gonna review it this cycle | |
| 17:45:33 | _colby | but in our case Nova does not seem to be creating any mdevs | |
| 17:45:51 | bauzas | I need to go but shouldn't be | |
| 17:46:07 | _colby | no problem thanks | |
| 17:46:29 | bauzas | I can't remember when we added more logs to _allocate_mdevs() but yoga seems recent enough to have those | |
| 17:48:32 | bauzas | https://github.com/openstack/nova/commit/6feb3350b048606297068841e3feba110bb0b0ab | |
| 17:48:46 | bauzas | nah, antelope | |
| 17:49:41 | _colby | I can manually patch our system to add these logs for testing | |
| 18:19:47 | _colby | bauzas: whenever you get back I made more progress: https://pastebin.com/Xv2bvaf7 | |
| 18:20:24 | _colby | its compaining about no available instances on that device but there is: cat /sys/class/mdev_bus/0000\:17\:02.0/mdev_supported_types/nvidia-563/available_instances | |
| 18:22:22 | _colby | also that device is not listed in the config for that type: device_addresses=0000:17:00.4,0000:17:00.5,0000:17:00.6,0000:17:00.7 | |
| 21:51:51 | opendevreview | Merged openstack/nova master: Allow config to support virtiofs (driver) https://review.opendev.org/c/openstack/nova/+/886522 | |
| 21:51:59 | opendevreview | Merged openstack/nova master: Attach Manila shares via virtiofs (db) https://review.opendev.org/c/openstack/nova/+/831193 | |
| #openstack-nova - 2024-01-19 | |||
| 00:36:34 | opendevreview | Takashi Natsume proposed openstack/nova-specs master: Create specs directory for 2024.2 Dalmatian https://review.opendev.org/c/openstack/nova-specs/+/906073 | |
| 05:23:47 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Updates glance fixture for create image https://review.opendev.org/c/openstack/nova/+/906088 | |
| 05:23:48 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/906089 | |
| 05:44:49 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 05:44:49 | opendevreview | Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648 | |
| 05:44:50 | opendevreview | Amit Uniyal proposed openstack/nova master: Removed explicit call to delete attachment https://review.opendev.org/c/openstack/nova/+/891289 | |
| 08:32:35 | bauzas | _colby: I'm here now but let's discuss this when you're back, I have some questions (saw your pastebin) | |
| 09:19:05 | opendevreview | Amit Uniyal proposed openstack/nova stable/2023.2: Fixes: bfv vm reboot ends up in an error state. https://review.opendev.org/c/openstack/nova/+/906089 | |
| 09:55:03 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Configure and teardown ephemeral encryption secrets https://review.opendev.org/c/openstack/nova/+/826754 | |
| 09:55:04 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to convert_image https://review.opendev.org/c/openstack/nova/+/870934 | |
| 09:55:04 | opendevreview | melanie witt proposed openstack/nova master: imagebackend: Add support to libvirt_info for LUKS based encryption https://review.opendev.org/c/openstack/nova/+/826755 | |
| 09:55:05 | opendevreview | melanie witt proposed openstack/nova master: Support resize with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933 | |
| 09:55:05 | opendevreview | melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932 | |
| 09:55:06 | opendevreview | melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936 | |
| 09:55:06 | opendevreview | melanie witt proposed openstack/nova master: Add hw_ephemeral_encryption_secret_uuid image property https://review.opendev.org/c/openstack/nova/+/870935 | |
| 09:55:07 | opendevreview | melanie witt proposed openstack/nova master: Support rebuild and unshelve with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/870939 | |
| 09:55:07 | opendevreview | melanie witt proposed openstack/nova master: Support snapshot with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870937 | |
| 09:55:09 | opendevreview | melanie witt proposed openstack/nova master: libvirt: make |
|
| 09:55:09 | opendevreview | melanie witt proposed openstack/nova master: Support rescue with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/873675 | |
| 09:55:11 | opendevreview | melanie witt proposed openstack/nova master: Reject (resize|rebuild) API requests with conflicting encryption https://review.opendev.org/c/openstack/nova/+/904240 | |
| 09:55:11 | opendevreview | melanie witt proposed openstack/nova master: WIP Support migration with ephemeral encryption https://review.opendev.org/c/openstack/nova/+/905512 | |
| 09:55:13 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for raw with LUKS https://review.opendev.org/c/openstack/nova/+/884313 | |
| 09:55:13 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | |
| 09:55:15 | opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for rbd with LUKS https://review.opendev.org/c/openstack/nova/+/889912 | |
| 10:46:02 | opendevreview | Sylvain Bauza proposed openstack/nova master: vgpu: Allow device_addresses to not be set https://review.opendev.org/c/openstack/nova/+/902084 | |
| 10:46:02 | opendevreview | Sylvain Bauza proposed openstack/nova master: libvirt: Cap with max_instances GPU types https://review.opendev.org/c/openstack/nova/+/899625 | |
| 10:46:03 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP: add coverage for registered dyn opts https://review.opendev.org/c/openstack/nova/+/902771 | |
| 10:54:54 | opendevreview | Sylvain Bauza proposed openstack/nova master: FUP: add coverage for registered dyn opts https://review.opendev.org/c/openstack/nova/+/902771 | |
| 13:18:46 | opendevreview | Tobias Urdin proposed openstack/nova master: Fail nova-compute startup when RP cannot be created https://review.opendev.org/c/openstack/nova/+/904381 | |
| 13:25:04 | opendevreview | Tobias Urdin proposed openstack/nova master: libvirt: set remaining to 0 when no disk to migrate https://review.opendev.org/c/openstack/nova/+/873846 | |
| 13:25:09 | opendevreview | Tobias Urdin proposed openstack/nova master: libvirt: set remaining to 0 when no disk to migrate https://review.opendev.org/c/openstack/nova/+/873846 | |
| 13:29:28 | opendevreview | Tobias Urdin proposed openstack/nova master: Fix wrong nova-manage command in upgrade check https://review.opendev.org/c/openstack/nova/+/880819 | |
| 13:33:50 | opendevreview | Tobias Urdin proposed openstack/nova master: Remove libvirt tunnelled migration https://review.opendev.org/c/openstack/nova/+/879021 | |
| 13:36:20 | opendevreview | Tobias Urdin proposed openstack/nova master: Remove libvirt tunnelled migration https://review.opendev.org/c/openstack/nova/+/879021 | |
| 14:28:45 | opendevreview | Tobias Urdin proposed openstack/nova master: Fail nova-compute startup when RP cannot be created https://review.opendev.org/c/openstack/nova/+/904381 | |
| 14:38:30 | opendevreview | Tobias Urdin proposed openstack/nova master: Fail nova-compute startup when RP cannot be created https://review.opendev.org/c/openstack/nova/+/904381 | |
| 14:48:43 | opendevreview | Merged openstack/nova master: [ironic] Partition & use cache for list_instance* https://review.opendev.org/c/openstack/nova/+/900831 | |
| 14:52:32 | bauzas | gibi: thanks a lot for the mdev series on both bugfix series | |
| 14:52:47 | bauzas | gibi: are you available for discussing about https://review.opendev.org/c/openstack/nova/+/845757/3/nova/virt/libvirt/driver.py#8563 ? | |
| 14:58:55 | bauzas | (just replied) | |
| 14:59:17 | gibi | bauzas: o/. | |
| 14:59:34 | bauzas | tl;dr: we continue to support VGPU allocations reshape in our code | |
| 14:59:40 | bauzas | and yeah, that's crazy. | |
| 14:59:45 | gibi | so the reshape test needs a from state and to create a from state we need to support some old behavior? | |
| 15:00:07 | bauzas | gibi: well, the better would be to cut from tree all the reshape code | |
| 15:00:15 | bauzas | that's what I replied | |
| 15:00:30 | bauzas | I'm just about to file a blueprint about it | |
| 15:01:37 | bauzas | papework-wise, I feel brave enough to have a 6th effort on VGPU (and a 6th potential conflict) that would drop the whole reshape code (and tests) | |
| 15:01:50 | bauzas | and then amend my bugfix to not care about non-nested RPs | |
| 15:02:08 | bauzas | but see, that's chicken and eggs here | |
| 15:02:09 | gibi | can we decouple the test from the reshape support? E.g. can we prepare the from state purely in placement / db and then let the reshape code run on that state? | |
| 15:02:53 | gibi | (I'm not suggesting to do the reshape removal in the current cycle) | |
| 15:03:26 | bauzas | gibi: not sure I get your point, are you asking me to modify my patch to assert that that the allocated RP is always a child RP ? | |
| 15:03:51 | bauzas | and then, just fix the test to not fail | |
| 15:03:54 | gibi | 1) on your patch I only asked for a tracker for the cleanup so you don't need to change anything there | |
| 15:04:32 | gibi | 2) when thinking about the removal of the `allocated_rp.parent_uuid is None:` conditional | |
| 15:05:13 | gibi | I assume that you need that conditional becuase the functional reshape test sets up the from state (with VGPU on root RP) while nova is running and therefore you need the conditional | |
| 15:05:28 | gibi | is it so? | |
| 15:06:51 | gibi | OR, does _allocate_mdevs called during reshape itself? | |