| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-18 | |||
| 08:53:18 | bauzas | gibi: why ? | |
| 08:53:22 | gibi | did we do that before in other stable branches? | |
| 08:53:30 | lyarwood | gibi: yeah we've done that in the past | |
| 08:53:42 | stephenfin | I guess we'll change the version tested if we backport, yes. We won't change the versions allowed though | |
| 08:53:43 | lyarwood | bauzas: grenade | |
| 08:53:45 | gibi | lyarwood: so it is sort of a normal thing at a min bump? | |
| 08:53:48 | brinzhang_ | gibi: IMO, in V release is enough | |
| 08:54:03 | lyarwood | gibi: iirc the last time we did this was due to QEMU/libvirt bugs | |
| 08:54:16 | bauzas | lyarwood: what? | |
| 08:54:24 | lyarwood | gibi: so we enabled the UCA to workaround these and had to do it on an older stable release for grenade as well | |
| 08:54:31 | bauzas | lyarwood: I thought grenade was upgrading on the target | |
| 08:54:42 | brinzhang_ | Cyborg done this today https://review.opendev.org/#/q/project:openstack/cyborg+branch:master+topic:migrate-to-focal | |
| 08:54:43 | lyarwood | bauzas: it does but it doesn't rerun devstack | |
| 08:54:54 | brinzhang_ | bump to the min version | |
| 08:55:06 | lyarwood | bauzas: so we'd need to use the train UCA on stable/ussuri to ensure n-cpu from master would work | |
| 08:55:23 | lyarwood | stephenfin: correct, it's just the tested version in stable/ussuri being changed | |
| 08:55:42 | lyarwood | and again I don't think anyone is using the min versions of libvirt and QEMU at the moment | |
| 08:56:06 | lyarwood | if you're using Ubuntu you're using the UCAs | |
| 08:56:20 | lyarwood | if you're using el distro you;re already way past these versions on stable/ussuri | |
| 08:56:56 | bauzas | ok, either way | |
| 08:57:04 | bauzas | if we merge the bumps | |
| 08:57:28 | bauzas | are we sure that the new minimums are satisfied by the train UCA ? | |
| 08:57:40 | bauzas | because | |
| 08:57:50 | bauzas | we could just don't care about train UCA | |
| 08:57:59 | lyarwood | bauzas: yes see https://review.opendev.org/#/c/746981/ | |
| 08:58:12 | bauzas | but if we would like to tackle the other issue, we would need to deploy a Bionic with Train UCA | |
| 08:58:23 | lyarwood | bauzas: specifically https://zuul.opendev.org/t/openstack/build/318b52b1aaf6473787405c9ed1ca0bc0/log/controller/logs/dpkg-l.txt | |
| 08:58:30 | bauzas | perfect, then I'm clear | |
| 08:58:37 | lyarwood | bauzas: that change Depends-On the devstack change pulling in the train UCA | |
| 08:58:50 | lyarwood | bauzas: all of the non grenade jobs are working | |
| 08:58:59 | gibi | OK, so it is not unprecedentet to change the tested libvirt version on a stable branch | |
| 08:59:00 | bauzas | okay, so I personnally feel that bumping minimums are reasonably safe | |
| 08:59:02 | gibi | so lets do it then | |
| 08:59:09 | bauzas | +1 | |
| 08:59:16 | gibi | bump the minimum and turn on train UCA in stable/ussuri | |
| 08:59:40 | bauzas | and let's do what we approved yesterday for the Focal thing | |
| 08:59:51 | bauzas | the gmann's work on jobs | |
| 09:00:08 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.moves https://review.opendev.org/727224 | |
| 09:00:21 | bauzas | lyarwood: sean-k-mooney: that being said, we need to document the gap | |
| 09:00:47 | openstack | Launchpad bug 1882521 in OpenStack Compute (nova) "Failing device detachments on Focal" [High,Confirmed] - Assigned to Lee Yarwood (lyarwood) | |
| 09:00:47 | bauzas | I guess we can use https://bugs.launchpad.net/nova/+bug/1882521 as a base start | |
| 09:01:11 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757 | |
| 09:01:12 | gibi | bauzas: could you drop the -2 from https://review.opendev.org/#/c/749707 then? | |
| 09:01:22 | bauzas | gibi: I was on it | |
| 09:01:25 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777 | |
| 09:01:27 | gibi | coolio | |
| 09:01:28 | lyarwood | ubuntu-- https://zuul.opendev.org/t/openstack/build/fc4fd2d6fcd14618899060c4e84edca0/log/job-output.txt#6579 one small thing to fix here for the evacuation tests also | |
| 09:01:44 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094 | |
| 09:01:52 | lyarwood | bauzas: and you mean outside of https://review.opendev.org/#/c/746981/6/releasenotes/notes/victoria-libvirt-version-bump-e1a09b3a72ee56a4.yaml | |
| 09:01:58 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109 | |
| 09:02:24 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (2/2) https://review.opendev.org/728117 | |
| 09:02:59 | openstackgerrit | Takashi Natsume proposed openstack/nova master: doc: correct the link to user/flavor.rst https://review.opendev.org/740176 | |
| 09:03:09 | bauzas | lyarwood: no, it's a matter of communicating the caveat we found with Focal | |
| 09:03:16 | openstackgerrit | Takashi Natsume proposed openstack/nova master: doc: Fix rendering in the accelerator support doc https://review.opendev.org/751574 | |
| 09:03:29 | bauzas | because TC and the Foundation would communicate on the fact that service projects do support Focal | |
| 09:03:36 | bauzas | but we regress on a thing | |
| 09:03:46 | bauzas | so, ideally, I would comment this in a prelude section | |
| 09:04:08 | lyarwood | bauzas: ah sorry right can we include a known issues section? | |
| 09:04:13 | bauzas | lyarwood: I guess that RHEL-based distros are also impacted by the volume attach issue since it's a qemu problem ? | |
| 09:04:23 | bauzas | lyarwood: that'd be the right position I guess | |
| 09:05:47 | bauzas | lyarwood: reno supports an 'issues' section https://docs.openstack.org/reno/latest/user/usage.html#editing-a-release-note | |
| 09:06:16 | lyarwood | bauzas: I've not been able to reproduce it on el distros yet | |
| 09:06:28 | bauzas | ah shit | |
| 09:06:38 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_FILE_BACKED_DISCARD_VERSION https://review.opendev.org/746982 | |
| 09:06:38 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION https://review.opendev.org/746981 | |
| 09:06:39 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_BETTER_SIGKILL_HANDLING https://review.opendev.org/746984 | |
| 09:06:39 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_{LIBVIRT,QEMU}_NATIVE_TLS_VERSION https://review.opendev.org/746983 | |
| 09:06:40 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_{LIBVIRT,QEMU}_PMEM_SUPPORT https://review.opendev.org/746986 | |
| 09:06:40 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_VIDEO_MODEL_VERSIONS https://review.opendev.org/746985 | |
| 09:06:41 | openstackgerrit | Lee Yarwood proposed openstack/nova master: test_evacuate.sh: Support libvirt-bin and libvirtd systemd services https://review.opendev.org/752650 | |
| 09:06:42 | bauzas | so it's not purely a qemu thing | |
| 09:07:38 | lyarwood | we believe it is but it also appears to be caused by overall load on the host etc so it's super awkward to reproduce | |
| 09:08:35 | bauzas | gibi: removed the -2 and +W the base patch | |
| 09:09:03 | bauzas | lyarwood: what would be a recommendation for operators using Focal ? | |
| 09:09:17 | bauzas | if we document a known issue | |
| 09:09:39 | bauzas | I mean, is there any found workaround with Focal ? | |
| 09:09:55 | lyarwood | no, there's not. | |
| 09:09:59 | lyarwood | they could hit this issue | |
| 09:10:03 | bauzas | or should we document as a known issue 'hold your OS upgrade if you consider running Victoria" | |
| 09:10:30 | lyarwood | well that's the thing we can't if we drop support for bionic | |
| 09:10:39 | bauzas | lyarwood: can't they try to override the base qemu versions by downgrading them using a Train UCA ? | |
| 09:10:40 | lyarwood | if we support both then we can say that | |
| 09:10:57 | lyarwood | bauzas: I doubt ubuntu would support that for a few release tbh | |
| 09:11:05 | bauzas | lyarwood: I'm in favor of continuing to support Bionic for that exact reason | |
| 09:11:30 | bauzas | this whole thing sucks. | |
| 09:11:40 | lyarwood | bauzas: if we can it would make our lives alot easier | |
| 09:12:36 | bauzas | honestly, we would then have to write "if you wanna use Victoria and you feel you're impacted by the bug, please consider running another distro that's not impacted" ? | |
| 09:12:56 | bauzas | this is absolutely serious. | |
| 09:13:28 | bauzas | can't we try to find Canonical engineers to help us ? | |
| 09:13:44 | bauzas | sahid isn't here AFAICS | |
| 09:13:59 | bauzas | jamespage neither. | |
| 09:14:03 | stephenfin | sahid left Canonical and is now...somewhere else | |
| 09:14:04 | lyarwood | kashyap has been trying to engage their QEMU engineer | |
| 09:14:15 | lyarwood | yes I did mean engineer | |
| 09:14:18 | lyarwood | 8| | |
| 09:14:52 | bauzas | gibi: have you engaged with the TC about this very critical issue ? | |
| 09:15:04 | lyarwood | brb I need more coffee | |
| 09:15:14 | bauzas | hopefully gmann did the liaison | |
| 09:15:39 | kashyap | lyarwood: Is this the Focal topic? | |
| 09:16:25 | kashyap | stephenfin: gibi: lyarwood: In general: in the past I've gone to Ubuntu's -stable IRC channel to point out missing virt-related backports | |