| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-07-02 | |||
| 14:35:03 | mnaser | dansmith: sorry I know its a busy morning but I was working on back porting the patches to unmaintained/zed | |
| 14:35:11 | mnaser | https://review.opendev.org/c/openstack/nova/+/923291/1/nova/tests/unit/virt/libvirt/test_utils.py | |
| 14:35:30 | mnaser | I think this will fail pep8 because of lack of new line above test_create_image_encryption | |
| 14:37:14 | dansmith | yep, looks like it already did | |
| 14:37:15 | dansmith | thanks | |
| 14:37:39 | dansmith | that was last minute yesterday | |
| 14:37:49 | mnaser | yeah it's all been last minute all over the place | |
| 14:37:52 | dansmith | it passed pep8 on master when I tried, so maybe the backport trimmed | |
| 14:38:02 | dansmith | srsly | |
| 14:38:07 | mnaser | I am taking care of zed for all the projects | |
| 14:38:18 | mnaser | https://review.opendev.org/c/openstack/nova/+/923297 if you want to eye it over (and it matters to you that is :p) | |
| 14:38:26 | sean-k-mooney | it passed for me locally too | |
| 14:38:47 | sean-k-mooney | i think its because in the gate its applied to the tip of the branch | |
| 14:38:59 | sean-k-mooney | when we run the jobs | |
| 14:39:06 | mnaser | ahh right maybe it ran it over the last commit that was clean | |
| 14:39:07 | sean-k-mooney | so to see it locally we might need to rebase | |
| 14:45:28 | tkajinam | I downloaded the chain for master but pep8 fails in my local (even after rebasing the chain on the latest master). zuul hasn't run pep8 year over 923258 but I suspect it may detect the same error. | |
| 14:45:40 | tkajinam | s/year/yet/ | |
| 14:46:02 | dansmith | oh it does fail for me on master, | |
| 14:46:09 | dansmith | I ran preivously before the tests were added | |
| 14:46:11 | dansmith | I will fix | |
| 14:47:07 | opendevreview | Dan Smith proposed openstack/nova master: Fix vmdk_allowed_types checking https://review.opendev.org/c/openstack/nova/+/923258 | |
| 14:47:54 | sean-k-mooney | reappoved | |
| 14:50:02 | opendevreview | Dan Smith proposed openstack/nova stable/2024.1: Fix vmdk_allowed_types checking https://review.opendev.org/c/openstack/nova/+/923276 | |
| 14:55:26 | opendevreview | Dan Smith proposed openstack/nova stable/2023.2: Fix vmdk_allowed_types checking https://review.opendev.org/c/openstack/nova/+/923287 | |
| 14:58:43 | dansmith | mnaser: tkajinam sorry about that, thanks for the heads up 2023.1 coming in a sec | |
| 15:00:59 | opendevreview | Dan Smith proposed openstack/nova stable/2023.1: Fix vmdk_allowed_types checking https://review.opendev.org/c/openstack/nova/+/923291 | |
| 15:09:09 | JayF | melwitt: sean-k-mooney: re: the lock greening errors in python 3.12: https://github.com/eventlet/eventlet/issues/969 is what itamarst tracked it to. | |
| 15:12:11 | frickler | sean-k-mooney: dansmith: do you want us to promote any of the approved patches into gate or do you prefer for all check jobs to finish first? | |
| 15:13:03 | dansmith | frickler: definitely want to see the tests first | |
| 15:19:47 | sean-k-mooney | frickler: based on the quick review we have done we belive the patchs shoudl be correct however we need the wider coverage form check to confirm we are not regressing any edgecases. if we see a timeout ectra or other infa issues we can consier promotiign or requeing | |
| 15:22:17 | frickler | well I'm not talking about force-merging (yet), you'll have the same jobs running in the gate pipeline again (except n-v), but anyway, the decision is up to you | |
| 15:27:44 | opendevreview | Fabian Wiesel proposed openstack/nova-specs master: Lazy Metadata Loading in order to Reduce Server Load https://review.opendev.org/c/openstack/nova-specs/+/922201 | |
| 15:32:40 | tkajinam | dansmith, no problem, and thank you ! | |
| 15:54:00 | dansmith | definitely want to see all the scenarios run against this since its so much change and it's nearly impossible to do that ahead of time | |
| 15:54:16 | dansmith | if we hit obviously unrelated things I think promote/force is the tool we can use to expedite | |
| 15:54:22 | dansmith | thanks frickler for checking | |
| 15:56:26 | sean-k-mooney | im checkign now but the nova-live-migration job fails on the second patch | |
| 15:56:28 | sean-k-mooney | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7fc/923256/1/check/nova-live-migration/7fc79c6/ | |
| 15:56:29 | opendevreview | Fabian Wiesel proposed openstack/nova-specs master: Lazy Metadata Loading in order to Reduce Server Load https://review.opendev.org/c/openstack/nova-specs/+/922201 | |
| 15:56:31 | sean-k-mooney | the ceph one passed | |
| 15:56:52 | dansmith | hmm | |
| 15:57:07 | dansmith | that is odd or sure | |
| 15:57:09 | dansmith | *for | |
| 15:57:20 | dansmith | I have to do a medial thing in a few minutes, so I will be out a bit around there | |
| 15:57:24 | sean-k-mooney | looks like only the blockmigation tests | |
| 15:57:38 | dansmith | right but perhaps something is upset about the disk image? | |
| 15:57:56 | dansmith | Jul 02 15:17:20.783035 np0037859393 nova-compute[41928]: ERROR oslo_messaging.rpc.server nova.exception.ImageUnacceptable: Image 02acc766-7c4b-4ffb-a72d-080ea50527d4 is unacceptable: Image not in a supported format | |
| 15:57:57 | dansmith | eff | |
| 15:58:19 | sean-k-mooney | yep | |
| 15:58:57 | sean-k-mooney | AttributeError: 'NoneType' object has no attribute 'from_file' | |
| 15:59:03 | sean-k-mooney | but i tought we caught that | |
| 16:01:15 | sean-k-mooney | so this is raising https://review.opendev.org/c/openstack/nova/+/923256/1/nova/virt/images.py#152 | |
| 16:01:32 | sean-k-mooney | and we take this else https://review.opendev.org/c/openstack/nova/+/923256/1/nova/virt/images.py#174 | |
| 16:02:07 | sean-k-mooney | oh ok i need to look at the ohte rhost to see it properly | |
| 16:04:12 | sean-k-mooney | Jul 02 15:19:58.679517 np0037859379 nova-compute[85178]: WARNING nova.virt.images [None req-3f27d1d1-b0a1-46a9-ac4b-6b869e49ea15 tempest-LiveMigrationTest-599041496 tempest-LiveMigrationTest-599041496-project-admin] Unable to perform deep image inspection on type 'aki' | |
| 16:04:28 | sean-k-mooney | so we have special cased for ami image but not aki | |
| 16:05:17 | sean-k-mooney | so i think its passing on the ami iamge and failing on the kernel image | |
| 16:06:06 | sean-k-mooney | dansmith: and we are only seeing it in that job becasue tis the only one using uec images | |
| 16:06:08 | sean-k-mooney | https://github.com/openstack/nova/blob/master/.zuul.yaml#L120-L121 | |
| 16:07:20 | sean-k-mooney | so "if disk_format == 'ami':" should be "if disk_format in ('ami','aki'):" | |
| 16:07:29 | sean-k-mooney | here https://review.opendev.org/c/openstack/nova/+/923256/1/nova/virt/images.py#166 | |
| 16:14:08 | dansmith | sean-k-mooney: aha, so I think we need to have a serious discussion about AMI in nova because as far as I can tell, the disk image can be anything from a qcow to a raw ext2 or ntfs filesystem and trying to determine safety there is a nightmare | |
| 16:14:10 | sean-k-mooney | dansmith: im carefully going to put a patch on the end to fix 'aki' | |
| 16:14:27 | sean-k-mooney | ya | |
| 16:14:32 | dansmith | perhaps if we fix our own losing track of the format of the base images in nova we could handle it better, but idk | |
| 16:14:37 | dansmith | sean-k-mooney: ack on the fix thanks ++ | |
| 16:14:40 | sean-k-mooney | i was also wonderign about the other contaienr formats like ova ovf | |
| 16:15:07 | sean-k-mooney | so we will need to squash the fix into patch 2 | |
| 16:15:17 | sean-k-mooney | but i dont want to kick the current set out of the gate | |
| 16:15:55 | dansmith | okay, yeah, let's confirm it fixes it and there's nothing else before we squash but sounds good | |
| 16:15:55 | sean-k-mooney | hench put it on the end and we can either force merge or pull it back depending on perfernce/urgency | |
| 16:15:59 | dansmith | we'll have to fix all the backports too | |
| 16:16:33 | sean-k-mooney | i sould aslo includ ari | |
| 16:16:56 | sean-k-mooney | https://creodias.docs.cloudferro.com/en/latest/cloud/What-Image-Formats-are-available-in-OpenStack-Creodias-Cloud.html | |
| 16:20:07 | dansmith | do we ever run qemu-img cmmands on the ramdisk? I'd guess not | |
| 16:20:51 | opendevreview | sean mooney proposed openstack/nova master: add support for aki and ari images https://review.opendev.org/c/openstack/nova/+/923317 | |
| 16:21:37 | sean-k-mooney | dansmith: i dont think we do but if we do its failign before we get to that point | |
| 16:22:07 | dansmith | maybe we just run it on all three pieces for no reason? but okay, no complaints about the patch as-is | |
| 16:22:52 | sean-k-mooney | proably realistically if any of the aobve were qcows i would expect it to trigger the vulnerablity | |
| 16:23:02 | sean-k-mooney | so checking all 3 part is a good thing to do regardless | |
| 16:23:23 | dansmith | ah, that's a good point | |
| 16:25:02 | sean-k-mooney | brb ill keep an eye on the jobs when im back | |
| 16:26:40 | gibi | I'm arond until the top of the hour (already +2 on the Sean's patch) | |
| 16:46:04 | sean-k-mooney | openstacksdk-functional-devstack failed on the 4 patch but is passing on the rest | |
| 16:46:07 | sean-k-mooney | enstack.exceptions.ConflictException: ConflictException: 409 | |
| 16:46:13 | sean-k-mooney | in the TestAutoAllocatedTopology test | |
| 16:46:26 | sean-k-mooney | i have seen that fail on othe rpatches and it seams to just be flaky | |
| 16:46:47 | sean-k-mooney | so for now ill leave the curernt set complete | |
| 16:47:25 | sean-k-mooney | it will get rerun once i rebase the patches for the ami/aki/ari fix anyway | |
| 16:47:46 | dansmith | ack | |
| 16:48:31 | sean-k-mooney | im assumign that merged reletivly recently because i have only started to see it fail like that in the last 2-3 weeks | |
| 16:48:41 | sean-k-mooney | i dont recall that job failing previously to that | |
| 16:50:43 | sean-k-mooney | hum no apprently that was added 8 years ago | |
| 16:51:05 | sean-k-mooney | that makes me feel old given the autoallocate stuff was added to enable people to move form nova-networks... | |
| 16:51:53 | sean-k-mooney | ofcouse everythign pre covide feels ancient at this point | |
| 17:40:53 | dansmith | sean-k-mooney: going to need a very quick approval on a job change in nova's ceph job to unblock glance in a sec | |
| 17:45:50 | opendevreview | Dan Smith proposed openstack/nova master: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923322 | |
| 17:46:03 | dansmith | sean-k-mooney: ^ | |
| 17:48:38 | dansmith | or any other nova cores | |