| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-23 | |||
| 13:50:04 | bauzas | wow | |
| 13:50:13 | bauzas | python-novaclient was left alone in the dark https://launchpad.net/python-novaclient/ | |
| 13:50:15 | sean-k-mooney | normally we just have it tracked on trun and then the backport but ya | |
| 13:57:39 | bauzas | ok, fixed on the client too | |
| 13:58:20 | gibi | bauzas: yes, I did the paperwork for yoga in LP. | |
| 13:58:35 | bauzas | I flipped the series | |
| 13:58:52 | bauzas | + I cleaned up novaclient which was needing some treatment | |
| 14:00:11 | gibi | bauzas: thanks | |
| 14:00:36 | bauzas | no worries, I had this too :) | |
| 14:07:17 | sean-k-mooney | bauzas: normally we set the current development focus to trunk | |
| 14:07:42 | sean-k-mooney | and only use yoga after its released for backporting | |
| 14:07:46 | bauzas | sean-k-mooney: that's not what the PTL guide says | |
| 14:08:03 | bauzas | https://docs.openstack.org/nova/latest/contributor/ptl-guide.html#immediately-after-rc | |
| 14:08:13 | bauzas | "Set the development focus of the project to the new cycle series" | |
| 14:08:14 | sean-k-mooney | that what we had previously doen or evey open bug on mastere would have to be taked as yoga | |
| 14:08:32 | bauzas | I can ask the release team to confirm | |
| 14:08:48 | sean-k-mooney | it might work either way if you dont have a trunk seriese | |
| 14:09:11 | sean-k-mooney | ya nova and neutron dont have trunk | |
| 14:09:37 | sean-k-mooney | os-vif does and we never have to update it just add the series | |
| 14:09:47 | sean-k-mooney | honestly its fine either way | |
| 14:09:52 | bauzas | sean-k-mooney: we do have a trunk series | |
| 14:09:54 | bauzas | https://launchpad.net/nova/+series | |
| 14:10:14 | bauzas | but it became obsolete | |
| 14:10:25 | sean-k-mooney | yes | |
| 14:10:26 | bauzas | looks to me we no longer need it | |
| 14:11:53 | bauzas | sean-k-mooney: guess what ? I'll stick with the yoga series :p | |
| 14:12:13 | sean-k-mooney | hehe thats fine by me | |
| 14:12:31 | sean-k-mooney | it proably makes working with blueprints simpler anyway | |
| 14:20:45 | gibi | bauzas: yepp, I always followed the guide and it worked for me to set the ongogin cycle as current release | |
| 14:22:24 | bauzas | ack | |
| 14:57:47 | bauzas | gibi: good luck with finding a race | |
| 14:57:54 | bauzas | hint : use GMR maybe | |
| 17:40:02 | opendevreview | Lee Yarwood proposed openstack/nova master: WIP compute: Update volume_id within connection_info during swap_volume https://review.opendev.org/c/openstack/nova/+/807025 | |
| 17:40:02 | opendevreview | Lee Yarwood proposed openstack/nova master: Add regression test for bug #1943431 https://review.opendev.org/c/openstack/nova/+/810755 | |
| 17:54:49 | melwitt | dansmith: wondered if you are familiar with errors like this in the nova-ceph-multistore job? "rbd.ReadOnlyImage: [errno 30] RBD read-only image (error creating snapshot b'nova-resize' from b'e261ad81-e30f-4ac7-a22c-1e4fd2b87db5_disk')" https://zuul.opendev.org/t/openstack/build/b37b95d96d6a4ae0a6025e92b515cf80/log/controller/logs/screen-n-cpu.txt#40935 | |
| 17:55:17 | dansmith | nope, but I'd go looking for ceph errors I think | |
| 17:56:19 | dansmith | that's in code that is libvirt and rbd-specific AFAIK, so that's a ceph readonlyness, not a glance one I think | |
| 17:58:17 | melwitt | ack | |
| 18:02:35 | gibi | melwitt: we have a tracking bug for that error https://bugs.launchpad.net/nova/+bug/1932043 | |
| 18:03:00 | gibi | but I have no clue what is causing it | |
| 18:03:08 | melwitt | thanks gibi | |
| 18:14:55 | opendevreview | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1944759 https://review.opendev.org/c/openstack/nova/+/810763 | |
| 18:37:02 | gibi | bauzas: that was the race ^^ :) | |
| 19:23:15 | lyarwood | kashyap: https://zuul.opendev.org/t/openstack/build/0f9c492eb1f84c1fbde643a94896c878/log/controller/logs/screen-n-cpu.txt?severity=4#24297 - for the morning, can you take a look at this? I'll also ask in #virt. I'm seeing it on centos-8 stream CI runs after a while and instance creation starts failing. Doesn't appear to be due to memory pressure but I might be missing something. | |
| 20:57:35 | ade_lee | lyarwood, ping - hey - how familiar are you with the cryptsetup? | |
| 20:58:22 | lyarwood | ade_lee: yeah I know my way around, is this about test failures? | |
| 20:58:28 | ade_lee | lyarwood, specifically, I have a tempest test that is failing under fips | |
| 20:58:33 | ade_lee | yup | |
| 20:58:36 | ade_lee | https://zuul.opendev.org/t/openstack/build/86cd01338d4a47d1854776b52d696ec9/log/controller/logs/screen-n-cpu.txt#58389 | |
| 20:59:51 | ade_lee | it seems the os-brick is calling cryptsetup here which is failing because cryptsetup is using plain encryption and that uses ripemd160 by default -- which is invalid under fips | |
| 21:00:27 | lyarwood | ade_lee: tbh plain cryptsetup encryption has been deprecated for a while now | |
| 21:00:44 | lyarwood | ade_lee: so in terms of getting it to work here with FIPS I'm not entirely sure it's worth the effort | |
| 21:01:18 | ade_lee | lyarwood, ack - thats what I was thinking too -- wanted to confirm that we just wanted to skip the test | |
| 21:01:23 | lyarwood | https://github.com/openstack/os-brick/blob/4f63dd045ca07f66003d1ccbbb19423ee91cf926/os_brick/encryptors/cryptsetup.py#L137-L145 | |
| 21:01:54 | lyarwood | the only reason we haven't removed it yet is because of the forced retype in cinder | |
| 21:02:03 | lyarwood | and all of the fun that brings to the table | |
| 21:02:22 | ade_lee | yup - I was just reading your note before :) | |
| 21:03:00 | ade_lee | ok -- so then this is a caveat we'll need to call out in case someone is updating their system to use fips | |
| 21:03:33 | ade_lee | if you have any old volumes , you need to retype them | |
| 21:04:13 | lyarwood | ack yeah | |
| 21:05:20 | ade_lee | lyarwood, what about this then -- https://zuul.opendev.org/t/openstack/build/fdf2ddc92ae64021a1386b027c7eebcf/log/controller/logs/screen-c-vol.txt#16290 | |
| 21:06:53 | opendevreview | Lee Yarwood proposed openstack/nova master: Add regression test for bug #1943431 https://review.opendev.org/c/openstack/nova/+/810755 | |
| 21:06:54 | opendevreview | Lee Yarwood proposed openstack/nova master: compute: Update volume_id within connection_info during swap_volume https://review.opendev.org/c/openstack/nova/+/807025 | |
| 21:06:54 | opendevreview | Lee Yarwood proposed openstack/nova master: fup: Move _wait_for_volume_{attach,detach} to os-volume_attachments https://review.opendev.org/c/openstack/nova/+/810775 | |
| 21:06:55 | opendevreview | Lee Yarwood proposed openstack/nova master: fup: Refactor and simplify Cinder fixture GET volume mock https://review.opendev.org/c/openstack/nova/+/810776 | |
| 21:07:05 | lyarwood | ade_lee: you can ignore that | |
| 21:07:37 | ade_lee | lyarwood, eh? isn't it causing the test to fail? | |
| 21:07:45 | lyarwood | ade_lee: That's just an initial check of the format of the device, if it's not LUKS we reformat. | |
| 21:08:17 | lyarwood | it's not failing because of that | |
| 21:08:20 | lyarwood | trace req-e52cdd22-a649-4134-9a83-75b41fc5f2d3 | |
| 21:08:29 | lyarwood | the request continues, it's just poor logging from os-brick | |
| 21:09:37 | ade_lee | lyarwood, seems like the cryptsetup isLuks is failing coz of "Running in FIPS mode.\nCommand failed with code -1 (wrong or missing parameters)." | |
| 21:11:00 | ade_lee | lyarwood, not sure if thats because its not a luks device and we need to reformat or if the command simply wont work under fips | |
| 21:12:14 | lyarwood | $ sudo cryptsetup isLuks --verbose /dev/vda | |
| 21:12:14 | lyarwood | Command failed with code -1 (wrong or missing parameters). | |
| 21:12:29 | lyarwood | Yeah the second part is what we see for unencrypted disks | |
| 21:13:48 | lyarwood | the rest of the flow completes | |
| 21:14:21 | lyarwood | there's another error when trying to use multipathd but again that looks like a poorly logged issue in os-brick and nothing to do with fips | |
| 21:14:24 | lyarwood | Sep 08 17:16:49.575982 centos-8-stream-rax-dfw-0026368401 cinder-volume[109619]: INFO cinder.volume.manager [None req-e52cdd22-a649-4134-9a83-75b41fc5f2d3 tempest-TestEncryptedCinderVolumes-1784657899 None] Created volume successfully. | |
| 21:16:04 | lyarwood | but that said | |
| 21:16:05 | lyarwood | 2021-09-08 17:21:22,863 117508 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': ' |
|
| 21:16:13 | lyarwood | ^ that's dumped when we fail the test | |
| 21:16:20 | lyarwood | so the guestOS didn't boot | |
| 21:22:50 | lyarwood | ade_lee: Yeah this is weird, the volume creation looks fine but there's nothing being logged to the console of the instance | |
| 21:23:01 | lyarwood | ade_lee: You might want to ask the Cinder folks to look at this tbh | |
| 21:23:31 | lyarwood | from a Nova POV the instance boots correctly so the disk is at least currently encrypted with the expected passphrase etc | |
| 21:23:40 | lyarwood | correctly* | |
| 21:24:22 | ade_lee | lyarwood, ack thanks for looking. it is weird. | |
| #openstack-nova - 2021-09-24 | |||
| 00:26:32 | opendevreview | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | |
| 06:27:55 | frickler | good morning nova, it seems you are running the l-c job in gate as non-voting, which doesn't make sense to me, see e.g. https://review.opendev.org/809955 | |
| 07:31:06 | bauzas | good Friday, Nova | |
| 07:31:26 | bauzas | frickler: tell me, we had issues with this job due to some dep | |
| 07:32:04 | bauzas | frickler: saw the mailing thread from gibi ? can find it if you want | |
| 07:36:08 | bauzas | actually, the problem is on placement, not nova | |
| 07:39:52 | elodilles | well, the problem is there in several projects | |
| 07:40:05 | bauzas | elodilles: I don't disagree | |
| 07:40:17 | elodilles | bauzas: good to hear that :) | |
| 07:40:33 | bauzas | but I'd somehow appreciate that we could only make the job non-voting only when needed | |
| 07:40:56 | bauzas | for nova, ussuri and later provide decorator>=4.0.0 | |
| 07:40:57 | elodilles | and nova has it too in ussuri and older branches | |