| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-29 | |||
| 12:30:41 | gibi | thanks | |
| 12:31:34 | sean-k-mooney | it sound like we are just not creating the rescue disk in ceph but trying to use it | |
| 12:31:34 | sean-k-mooney | it sound like we are just not creating the rescue disk in ceph but trying to use it | |
| 12:32:16 | sean-k-mooney | i wonder is that becaue of the fact the vm is bfv. | |
| 12:32:17 | sean-k-mooney | i wonder is that becaue of the fact the vm is bfv. | |
| 12:34:20 | sean-k-mooney | they are on victoria so they should have https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/virt-bfv-instance-rescue.html | |
| 12:34:21 | sean-k-mooney | they are on victoria so they should have https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/virt-bfv-instance-rescue.html | |
| 12:35:52 | sean-k-mooney | my guess is that we are creating the rescue disk as a cinder volumn but then trying to use it form teh vms pool since the rbd images backend is enabeld | |
| 12:35:52 | sean-k-mooney | my guess is that we are creating the rescue disk as a cinder volumn but then trying to use it form teh vms pool since the rbd images backend is enabeld | |
| 12:43:58 | sean-k-mooney | so this is going to use the image_backed specified in the nova.conf so the disk it returns would be basedon the rbd image backend which will use the vms pool | |
| 12:43:58 | sean-k-mooney | so this is going to use the image_backed specified in the nova.conf so the disk it returns would be basedon the rbd image backend which will use the vms pool | |
| 12:44:00 | sean-k-mooney | https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L5170 | |
| 12:44:00 | sean-k-mooney | https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L5170 | |
| 12:47:16 | sean-k-mooney | _create_image is going too check if its a bfv guest here https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4500 | |
| 12:47:16 | sean-k-mooney | _create_image is going too check if its a bfv guest here https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4500 | |
| 12:48:36 | sean-k-mooney | which will be passed to _create_and_inject_local_root https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4578 | |
| 12:48:37 | sean-k-mooney | which will be passed to _create_and_inject_local_root https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4578 | |
| 12:49:25 | sean-k-mooney | which will not create teh image becuase bfv willl be true https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4664 | |
| 12:49:25 | sean-k-mooney | which will not create teh image becuase bfv willl be true https://github.com/openstack/nova/blob/eba9d596daa91d8f702b719afb88cb89f2d5bb32/nova/virt/libvirt/driver.py#L4664 | |
| 12:49:46 | sean-k-mooney | so we will just take the else branch | |
| 12:49:46 | sean-k-mooney | so we will just take the else branch | |
| 12:50:09 | sean-k-mooney | lyarwood: gibi ^ pretty sure that is the root cause of https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:50:10 | openstack | Launchpad bug 1926601 in OpenStack Compute (nova) "Rescuing RBD volume-backed instance does not work" [Undecided,New] | |
| 12:50:10 | sean-k-mooney | lyarwood: gibi ^ pretty sure that is the root cause of https://bugs.launchpad.net/nova/+bug/1926601 | |
| 12:50:10 | openstack | Launchpad bug 1926601 in OpenStack Compute (nova) "Rescuing RBD volume-backed instance does not work" [Undecided,New] | |
| 12:50:33 | gibi | sean-k-mooney: nice analysis | |
| 12:50:33 | gibi | sean-k-mooney: nice analysis | |
| 12:53:42 | lyarwood | hmm I was sure we had tempest tests for this | |
| 12:53:42 | lyarwood | hmm I was sure we had tempest tests for this | |
| 12:53:53 | lyarwood | and I also verified it during the bfv rescue work a few cycles ago | |
| 12:53:54 | lyarwood | and I also verified it during the bfv rescue work a few cycles ago | |
| 12:53:55 | lyarwood | * lyarwood looks | |
| 13:26:46 | lyarwood | sean-k-mooney: thanks for the pointers, the real issue is that the request was even allowed as we don't support bfv rescue outside of also requesting a stable device rescue. | |
| 13:26:46 | lyarwood | sean-k-mooney: thanks for the pointers, the real issue is that the request was even allowed as we don't support bfv rescue outside of also requesting a stable device rescue. | |
| 13:27:33 | lyarwood | appears I never encoded that in the compute API so it's accepted and passed down to the virt driver that was never changed to support this with legacy rescue attempts (rescue device first etc) | |
| 13:27:33 | lyarwood | appears I never encoded that in the compute API so it's accepted and passed down to the virt driver that was never changed to support this with legacy rescue attempts (rescue device first etc) | |
| 13:27:57 | lyarwood | stable device rescues of bfv instances works just fine and is tested in tempest | |
| 13:27:57 | lyarwood | stable device rescues of bfv instances works just fine and is tested in tempest | |
| 13:28:09 | lyarwood | I'll write up a regression test and add some logic in the compute API to avoid this | |
| 13:28:09 | lyarwood | I'll write up a regression test and add some logic in the compute API to avoid this | |
| 13:29:52 | lyarwood | https://docs.openstack.org/nova/latest/user/rescue.html#instance-rescue I did at least call it out in the docs | |
| 13:29:52 | lyarwood | https://docs.openstack.org/nova/latest/user/rescue.html#instance-rescue I did at least call it out in the docs | |
| 13:35:35 | sean-k-mooney | lyarwood: ok so for the bug i guess we coudl do two things. one explain how to use stabel rescue | |
| 13:35:35 | sean-k-mooney | lyarwood: ok so for the bug i guess we coudl do two things. one explain how to use stabel rescue | |
| 13:35:49 | sean-k-mooney | and second update it to track blocking it in the api with a 400 | |
| 13:35:49 | sean-k-mooney | and second update it to track blocking it in the api with a 400 | |
| 13:36:45 | lyarwood | yup indeed, I'll sort both out shortly | |
| 13:36:45 | lyarwood | yup indeed, I'll sort both out shortly | |
| 14:07:48 | lyarwood | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1926375 seen this? | |
| 14:07:49 | lyarwood | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1926375 seen this? | |
| 14:07:49 | openstack | Launchpad bug 1926375 in OpenStack Compute (nova) "nova-compute service failed to start up" [Undecided,New] | |
| 14:07:50 | openstack | Launchpad bug 1926375 in OpenStack Compute (nova) "nova-compute service failed to start up" [Undecided,New] | |
| 14:21:49 | sean-k-mooney | lyarwood: i can see why that would happen | |
| 14:21:50 | sean-k-mooney | lyarwood: i can see why that would happen | |
| 14:22:20 | sean-k-mooney | if the binding profile in the neutorn port was currupted and the pci_slot key was removed that would happen | |
| 14:22:20 | sean-k-mooney | if the binding profile in the neutorn port was currupted and the pci_slot key was removed that would happen | |
| 14:24:33 | gibi | sean-k-mooney: yeah, but we cannot really prove anyithing as the logs does not have an instance uuid or port uuid to map it back to the neutron state | |
| 14:24:33 | gibi | sean-k-mooney: yeah, but we cannot really prove anyithing as the logs does not have an instance uuid or port uuid to map it back to the neutron state | |
| 14:25:01 | sean-k-mooney | well we can see plugin='ovs',port_profile=VIFPortProfileOpenVSwitch | |
| 14:25:01 | sean-k-mooney | well we can see plugin='ovs',port_profile=VIFPortProfileOpenVSwitch | |
| 14:25:10 | sean-k-mooney | and .plug_hw_veb | |
| 14:25:10 | sean-k-mooney | and .plug_hw_veb | |
| 14:25:24 | sean-k-mooney | so this is hardware offloaded ovs which does not support trusted VF | |
| 14:25:24 | sean-k-mooney | so this is hardware offloaded ovs which does not support trusted VF | |
| 14:28:06 | gibi | ohh, so we only support trusted VF with the sriov agent? | |
| 14:28:06 | gibi | ohh, so we only support trusted VF with the sriov agent? | |
| 14:28:12 | gibi | I did not know that | |
| 14:28:12 | gibi | I did not know that | |
| 14:28:46 | sean-k-mooney | if it works with anything else its by acident wew never extended support to anything else | |
| 14:28:46 | sean-k-mooney | if it works with anything else its by acident wew never extended support to anything else | |
| 14:30:41 | gibi | it seems they are using it :) | |
| 14:30:41 | gibi | it seems they are using it :) | |
| 14:34:19 | sean-k-mooney | well im pretty sure that it wont work the way they intend | |
| 14:34:19 | sean-k-mooney | well im pretty sure that it wont work the way they intend | |
| 14:35:11 | sean-k-mooney | it might allow them to change the mac adress but ovs wont make the port promisous and the security group rules will still be enforced | |
| 14:35:11 | sean-k-mooney | it might allow them to change the mac adress but ovs wont make the port promisous and the security group rules will still be enforced | |
| 14:40:52 | sean-k-mooney | gibi: ok i have marked it as incomplete for now https://bugs.launchpad.net/nova/+bug/1926375 | |
| 14:40:54 | openstack | Launchpad bug 1926375 in OpenStack Compute (nova) "nova-compute service failed to start up" [Undecided,Incomplete] | |
| 14:40:54 | sean-k-mooney | gibi: ok i have marked it as incomplete for now https://bugs.launchpad.net/nova/+bug/1926375 | |
| 14:40:54 | openstack | Launchpad bug 1926375 in OpenStack Compute (nova) "nova-compute service failed to start up" [Undecided,Incomplete] | |
| 14:41:10 | gibi | sean-k-mooney: ack, thanks | |
| 14:41:10 | gibi | sean-k-mooney: ack, thanks | |
| 15:49:39 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix bond_mode enum 802.1ad -> 802.3ad https://review.opendev.org/c/openstack/nova/+/788790 | |
| 15:49:39 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix bond_mode enum 802.1ad -> 802.3ad https://review.opendev.org/c/openstack/nova/+/788790 | |
| 15:49:59 | gibi | sean-k-mooney: could you double check that I understood this right ^^ ? | |
| 15:50:00 | gibi | sean-k-mooney: could you double check that I understood this right ^^ ? | |
| 15:52:24 | gibi | nova meeting starts in 8 minutes in #openstack-meeting-3 | |
| 15:52:24 | gibi | nova meeting starts in 8 minutes in #openstack-meeting-3 | |
| 15:57:49 | gibi | dansmith: regarding yesterday service version check discussion. You said that you think the version alias is wrong. I've double checked and I don't see the problem. 54 was the first Wallaby service version. 52 is the first Victoria service version | |
| 15:57:49 | gibi | dansmith: regarding yesterday service version check discussion. You said that you think the version alias is wrong. I've double checked and I don't see the problem. 54 was the first Wallaby service version. 52 is the first Victoria service version | |
| 15:58:21 | sean-k-mooney | gibi: am sure | |
| 15:58:22 | sean-k-mooney | gibi: am sure | |
| 15:58:48 | openstackgerrit | Merged openstack/nova master: zuul: Replace grenade and nova-grenade-multinode with grenade-multinode https://review.opendev.org/c/openstack/nova/+/778885 | |
| 15:58:48 | openstackgerrit | Merged openstack/nova master: zuul: Replace grenade and nova-grenade-multinode with grenade-multinode https://review.opendev.org/c/openstack/nova/+/778885 | |
| 15:59:24 | openstackgerrit | Merged openstack/nova master: zuul: Remove nova-dsvm-multinode-base https://review.opendev.org/c/openstack/nova/+/778908 | |
| 15:59:25 | openstackgerrit | Merged openstack/nova master: zuul: Remove nova-dsvm-multinode-base https://review.opendev.org/c/openstack/nova/+/778908 | |
| 16:00:28 | lyarwood | \o/ | |
| 16:00:28 | lyarwood | \o/ | |
| 16:00:33 | sean-k-mooney | gibi: ya 802.3ad is lacp bonding 802.1 is QinQ | |
| 16:00:33 | sean-k-mooney | gibi: ya 802.3ad is lacp bonding 802.1 is QinQ | |