| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-29 | |||
| 10:18:08 | gibi | yepp the logs are from the old code | |
| 10:18:08 | gibi | yepp the logs are from the old code | |
| 10:18:19 | lyarwood | anyway it's rebased now | |
| 10:18:19 | lyarwood | anyway it's rebased now | |
| 10:18:21 | lyarwood | so hopefully | |
| 10:18:21 | lyarwood | so hopefully | |
| 10:18:37 | gibi | fingers crossed | |
| 10:18:37 | gibi | fingers crossed | |
| 10:19:18 | gibi | lyarwood: it is a grenade run, so I guess before the upgrade it have to run the old code from Wallaby | |
| 10:19:18 | gibi | lyarwood: it is a grenade run, so I guess before the upgrade it have to run the old code from Wallaby | |
| 10:19:34 | lyarwood | aaaaaaaah | |
| 10:19:34 | lyarwood | aaaaaaaah | |
| 10:19:37 | lyarwood | right | |
| 10:19:37 | lyarwood | right | |
| 10:19:42 | lyarwood | I didn't even think | |
| 10:19:42 | lyarwood | I didn't even think | |
| 10:19:55 | lyarwood | so is this still v to w? | |
| 10:19:55 | lyarwood | so is this still v to w? | |
| 10:20:05 | gibi | I think it is w to master | |
| 10:20:05 | gibi | I think it is w to master | |
| 10:20:13 | gibi | but we only merged the fix to master | |
| 10:20:13 | gibi | but we only merged the fix to master | |
| 10:20:19 | sean-k-mooney | oh ya | |
| 10:20:19 | sean-k-mooney | oh ya | |
| 10:20:58 | lyarwood | we did talk about backporting your change to wallaby | |
| 10:20:59 | lyarwood | we did talk about backporting your change to wallaby | |
| 10:21:08 | sean-k-mooney | well grenade shoudl be wallaby to xena | |
| 10:21:08 | sean-k-mooney | well grenade shoudl be wallaby to xena | |
| 10:21:15 | lyarwood | * lyarwood needs to jump on a call brb | |
| 10:21:36 | lyarwood | sean-k-mooney: yeah I think this is the live migration test between the two | |
| 10:21:36 | lyarwood | sean-k-mooney: yeah I think this is the live migration test between the two | |
| 10:21:40 | lyarwood | sean-k-mooney: with the compute still on w | |
| 10:21:40 | lyarwood | sean-k-mooney: with the compute still on w | |
| 10:22:59 | sean-k-mooney | back porting the change for libvirt events to wallaby would be nice form a downstrem perspecitve too | |
| 10:23:00 | sean-k-mooney | back porting the change for libvirt events to wallaby would be nice form a downstrem perspecitve too | |
| 10:23:53 | sean-k-mooney | we talked about trying that downstream but while i like the idea of doing that i was not personally sure we wanted to diverge form upstream in this case | |
| 10:23:53 | sean-k-mooney | we talked about trying that downstream but while i like the idea of doing that i was not personally sure we wanted to diverge form upstream in this case | |
| 10:24:56 | gibi | lyarwood: the delta between w and current master should be small, so I can fire up the cherry-picks | |
| 10:24:56 | gibi | lyarwood: the delta between w and current master should be small, so I can fire up the cherry-picks | |
| 10:29:46 | gibi | ahh there are already merge conflicts :/ | |
| 10:29:47 | gibi | ahh there are already merge conflicts :/ | |
| 10:36:13 | lyarwood | yeah that might be my fault sorry | |
| 10:36:13 | lyarwood | yeah that might be my fault sorry | |
| 10:36:32 | lyarwood | actually yeah it is | |
| 10:36:33 | lyarwood | actually yeah it is | |
| 10:36:51 | lyarwood | sean-k-mooney: yeah I did bring it up upstream as well as it's also a bugfix IMHO | |
| 10:36:52 | lyarwood | sean-k-mooney: yeah I did bring it up upstream as well as it's also a bugfix IMHO | |
| 10:37:52 | gibi | it is nobody's fault, we needed those fixes | |
| 10:37:52 | gibi | it is nobody's fault, we needed those fixes | |
| 11:08:30 | openstackgerrit | Vlad Gusev proposed openstack/nova master: libvirt: Abort live-migration job when monitoring fails https://review.opendev.org/c/openstack/nova/+/764435 | |
| 11:08:30 | openstackgerrit | Vlad Gusev proposed openstack/nova master: libvirt: Abort live-migration job when monitoring fails https://review.opendev.org/c/openstack/nova/+/764435 | |
| 11:10:56 | openstackgerrit | Daniel Bengtsson proposed openstack/nova master: Use the new type HostDomainOpt. https://review.opendev.org/c/openstack/nova/+/788240 | |
| 12:13:05 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Replace blind retry with libvirt event waiting in detach https://review.opendev.org/c/openstack/nova/+/788720 | |
| 12:13:06 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Move the guest.get_disk test to test_guest https://review.opendev.org/c/openstack/nova/+/788721 | |
| 12:13:07 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Enable mypy on libvirt/guest.py https://review.opendev.org/c/openstack/nova/+/788722 | |
| 12:13:08 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Follow up type hints for a634103 https://review.opendev.org/c/openstack/nova/+/788723 | |
| 12:13:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: libvirt: Remove dead error handling code https://review.opendev.org/c/openstack/nova/+/788724 | |
| 12:13:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Move instance power state check to _detach_with_retry https://review.opendev.org/c/openstack/nova/+/788725 | |
| 12:14:24 | gibi | lyarwood, sean-k-mooney: ^^ here is the backport series. I'm not 100% confident about it as there was hairy conflicts in the tests | |
| 12:14:25 | gibi | lyarwood, sean-k-mooney: ^^ here is the backport series. I'm not 100% confident about it as there was hairy conflicts in the tests | |
| 12:14:51 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/wallaby: Consolidate device detach error handling https://review.opendev.org/c/openstack/nova/+/788726 | |
| 12:26:38 | gibi | lyarwood: fyi, there is a rescue + volume + RBD bug in our untriaged list https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:26:40 | openstack | Launchpad bug 1926601 in OpenStack Compute (nova) "Rescuing RBD volume-backed instance does not work" [Undecided,New] | |
| 12:26:40 | gibi | lyarwood: fyi, there is a rescue + volume + RBD bug in our untriaged list https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:29:04 | sean-k-mooney | this is hte rbd images backend | |
| 12:29:04 | sean-k-mooney | this is hte rbd images backend | |
| 12:29:31 | sean-k-mooney | and a bfv guest | |
| 12:29:32 | sean-k-mooney | and a bfv guest | |
| 12:29:55 | lyarwood | gibi: ack yeah just on a call but I'll triage it today | |
| 12:29:56 | lyarwood | gibi: ack yeah just on a call but I'll triage it today | |
| 12:30:41 | gibi | thanks | |
| 12:31:34 | sean-k-mooney | it sound like we are just not creating the rescue disk in ceph but trying to use it | |
| 12:31:34 | sean-k-mooney | it sound like we are just not creating the rescue disk in ceph but trying to use it | |
| 12:32:16 | sean-k-mooney | i wonder is that becaue of the fact the vm is bfv. | |
| 12:32:17 | sean-k-mooney | i wonder is that becaue of the fact the vm is bfv. | |
| 12:34:20 | sean-k-mooney | they are on victoria so they should have https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/virt-bfv-instance-rescue.html | |
| 12:34:21 | sean-k-mooney | they are on victoria so they should have https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/virt-bfv-instance-rescue.html | |
| 12:35:52 | sean-k-mooney | my guess is that we are creating the rescue disk as a cinder volumn but then trying to use it form teh vms pool since the rbd images backend is enabeld | |
| 12:35:52 | sean-k-mooney | my guess is that we are creating the rescue disk as a cinder volumn but then trying to use it form teh vms pool since the rbd images backend is enabeld | |
| 12:43:58 | sean-k-mooney | so this is going to use the image_backed specified in the nova.conf so the disk it returns would be basedon the rbd image backend which will use the vms pool | |
| 12:43:58 | sean-k-mooney | so this is going to use the image_backed specified in the nova.conf so the disk it returns would be basedon the rbd image backend which will use the vms pool | |
| 12:44:00 | sean-k-mooney | https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L5170 | |
| 12:44:00 | sean-k-mooney | https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L5170 | |
| 12:47:16 | sean-k-mooney | _create_image is going too check if its a bfv guest here https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4500 | |
| 12:47:16 | sean-k-mooney | _create_image is going too check if its a bfv guest here https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4500 | |
| 12:48:36 | sean-k-mooney | which will be passed to _create_and_inject_local_root https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4578 | |
| 12:48:37 | sean-k-mooney | which will be passed to _create_and_inject_local_root https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4578 | |
| 12:49:25 | sean-k-mooney | which will not create teh image becuase bfv willl be true https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4664 | |
| 12:49:25 | sean-k-mooney | which will not create teh image becuase bfv willl be true https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4664 | |
| 12:49:46 | sean-k-mooney | so we will just take the else branch | |
| 12:49:46 | sean-k-mooney | so we will just take the else branch | |
| 12:50:09 | sean-k-mooney | lyarwood: gibi ^ pretty sure that is the root cause of https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:50:10 | openstack | Launchpad bug 1926601 in OpenStack Compute (nova) "Rescuing RBD volume-backed instance does not work" [Undecided,New] | |
| 12:50:10 | sean-k-mooney | lyarwood: gibi ^ pretty sure that is the root cause of https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:50:10 | openstack | Launchpad bug 1926601 in OpenStack Compute (nova) "Rescuing RBD volume-backed instance does not work" [Undecided,New] | |
| 12:50:33 | gibi | sean-k-mooney: nice analysis | |
| 12:50:33 | gibi | sean-k-mooney: nice analysis | |
| 12:53:42 | lyarwood | hmm I was sure we had tempest tests for this | |
| 12:53:42 | lyarwood | hmm I was sure we had tempest tests for this | |
| 12:53:53 | lyarwood | and I also verified it during the bfv rescue work a few cycles ago | |