Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-07-02
19:52:30 dansmith I see in tempest config for neutron
19:52:36 dansmith I think that's right
19:53:08 opendevreview Dan Smith proposed openstack/nova master: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923322
19:53:14 abhishek_ hmm, ket me do that
19:53:40 fungi lmk as soon as it has an approval vote and i can reenqueue it into the gate
19:54:53 dansmith I guess this is why we don't (normally) skip check
19:55:37 abhishek_ ++
19:56:16 fungi heh
19:56:34 fungi we're not normally in quite so much of a hurry ;)
19:58:04 dansmith I'm want to get off the merry-go-round
19:59:16 fungi if we get it spinning fast enough, you can take advantage of centripetal force to make a speedy exit
19:59:51 fungi you'll probably land somewhere over by the cotton candy booth
20:01:35 sean-k-mooney given the week in it i expect a lot of people to dissapre after tomorrow one way or another
20:02:53 dansmith fungi: both of those are fixed (we think) and re-approved.. up to you if you want to take another gamble on the roulette wheel
20:03:49 dansmith sean-k-mooney: looks like unless ceph fails, everything passed except for that live migration job, which is fairly effing amazing
20:04:00 dansmith actually ceph already passed on the second one
20:04:00 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/923317/1
20:04:09 sean-k-mooney my fix passed
20:04:30 dansmith sweet, I'll squash here in a few when the ones below are done
20:04:30 sean-k-mooney and was green on all jobs
20:04:39 dansmith amazing
20:05:06 sean-k-mooney so ya you could also squash in the vmdk fix fi you wanted to reduce the set of patches
20:05:17 sean-k-mooney but ill leave that up to your judgement
20:05:34 sean-k-mooney part of me does not reallly want to mess with them if they are passing even if the end result is the same
20:06:07 fungi the new revisions of both changes are at the top of the gate pipeline again now
20:09:45 abhishek_ thank you!
20:23:05 sean-k-mooney we have resutls for https://review.opendev.org/c/openstack/nova/+/923256 now and the only failure was the aki/ari issue
20:23:33 sean-k-mooney tempest-integrated-compute-enforce-scope-new-defaults https://zuul.opendev.org/t/openstack/build/98a696b69cc844c6bdf9f7cb4b73f54d : SUCCESS in 2h 00m 47s
20:23:51 sean-k-mooney that job has a 2 hour timeout so it got very close
20:26:01 sean-k-mooney the last job on the 3rd patch is almost done too
20:28:45 sean-k-mooney ok thats done too
20:29:21 sean-k-mooney ill wait aorund for another 5 minutes otherwise ill check back in an hour or 2 before i go to sleep
20:29:32 opendevreview Dan Smith proposed openstack/nova master: Reject qcow files with data-file attributes https://review.opendev.org/c/openstack/nova/+/923255
20:29:32 opendevreview Dan Smith proposed openstack/nova master: Check images with format_inspector for safety https://review.opendev.org/c/openstack/nova/+/923256
20:29:33 opendevreview Dan Smith proposed openstack/nova master: Additional qemu safety checking on base images https://review.opendev.org/c/openstack/nova/+/923257
20:29:33 opendevreview Dan Smith proposed openstack/nova master: Fix vmdk_allowed_types checking https://review.opendev.org/c/openstack/nova/+/923258
20:30:08 sean-k-mooney im not sure you ment to update the first patch..
20:30:33 dansmith oh crap, yeah I rebased my origin master chasing the other fire
20:30:37 dansmith oh well, that sucks
20:30:58 dansmith it was rechecking anyway
20:31:50 sean-k-mooney i +2w'd the set
20:31:56 dansmith thanks much
20:32:12 dansmith I'm just about exhausted and fading pretty fast here
20:32:21 sean-k-mooney ok im going to go eat. ill check back in an hour or two before i go to sleep and ill check them in the morning if they have any issues
20:32:43 sean-k-mooney dansmith: go rest
20:32:43 dansmith I have one other thing to do and then I will probably need to take a break for a bit
20:33:04 sean-k-mooney well theere in the ci so not much more for you do do on the nova side
20:33:27 sean-k-mooney unless your plannig to respin the backports but we can do that once they start to merge
20:34:11 dansmith I'm going to hold off on those until we make progress yeah
22:08:37 fungi argh, one job timed out on the glance change
22:09:00 fungi during volume tests, though that may have been coincidence
22:10:11 fungi though the one remaining job on the nova change may be about to do the same
22:14:15 fungi they're both starting over at the top of the gate pipeline again
#openstack-nova - 2024-07-03
00:19:15 fungi yeah, seems like tempest-integrated-compute-enforce-scope-new-defaults is repeatedly timing out (during volume tests but those aren't necessarily what's taking so long) for both the glance and nova changes previously discussed
00:19:42 fungi though the glance one might barely finish just under the wire this time around
00:20:08 fungi once it (hopefully) merges, i'll reenqueue and promote the nova change again
00:25:11 fungi okay, glance change landed, nova change is back to the top of the gate for a third try
00:48:36 dansmith fungi++
00:54:54 dansmith fungi: that new defaults job has been timeout-heavy lately for sure, I think sean had a thing to bump the timeout, which I suppose we might have to do..
01:55:19 opendevreview Dan Smith proposed openstack/nova stable/2024.1: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923342
01:55:31 opendevreview Dan Smith proposed openstack/nova stable/2023.2: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923343
01:55:43 opendevreview Dan Smith proposed openstack/nova stable/2023.1: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923344
01:55:44 dansmith abhishek_: ^
01:56:24 dansmith melwitt: if you're around by chance and could monitor those ^ we need them in ASAP (master still pending) so we can get glance CVE patches merged that depend on them
01:56:41 abhishek_ dansmith: ++
02:12:11 melwitt dansmith: ok, I can do it
02:14:14 dansmith melwitt: thanks, master has been rechecked several times already, so who knows
02:16:51 melwitt yeah. we can at least get some more dice rolls in before EU timezone
02:21:16 abhishek_ is there someone in this timezone who can prioritise the patch in gate (just in case if required)
02:21:29 dansmith melwitt: ++
02:21:46 dansmith abhishek_: I'm not sure, frickler will be coming online, might be the earliest
02:21:58 abhishek_ ack
02:22:00 dansmith I think he can do it, but not sure
02:22:15 abhishek_ will ask him if required, thank you!
02:37:38 opendevreview Merged openstack/nova master: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923322
02:37:46 fungi yay!
02:37:51 dansmith amazing
02:38:18 dansmith fungi: do we prioritize the stable ones too or no?
02:39:17 fungi might not be necessary this time of day if they already have passing check results and there's not much else in the gate, i haven't looked
02:41:53 dansmith ack
02:43:25 fungi but if it will help in getting them merged faster we still can
02:43:54 fungi just depends on whether there's much of a queue or jobs are continuing to be problematic
02:44:00 dansmith abhishek_: ^ dunno how important it is
02:44:52 abhishek_ dansmith: I think we can wait, since it will going to take a day long to merge master patches
02:44:59 dansmith ack
02:46:17 abhishek_ nova merged
05:13:29 frickler well the gate is currently looking ok-ish after https://review.opendev.org/923344 caused a gate reset. let's hope the current stack merges without further failures. I'll try to keep an eye on it and look into reshuffling if further issues should happen
06:00:12 frickler and there goes hoping ... weird early failure in grenade-multinode that I haven't seen before, but looks 99% unrelated https://zuul.opendev.org/t/openstack/build/322833ee6e7a43b0aee2ca9803fc4f57
07:27:51 bauzas good morning folks
07:28:05 bauzas frickler: I was on PTO yesterday, so I just got the emails
07:28:16 bauzas but AFAICS, we needed to update Tempest rightN?
07:28:34 bauzas https://review.opendev.org/c/openstack/nova/+/923322
07:28:51 bauzas I'm now chasing https://review.opendev.org/c/openstack/nova/+/923255/2
07:32:28 frickler bauzas: the latter is failing in gate with the error I posted before, IMO it could be re-enqueued into gate immediately once zuul reports the failures, together with the stack on top of it
07:34:05 frickler I found keystone (once again) to be essentially undebuggable with the amount of errors generated during normal operations
07:34:58 bauzas yeah https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_322/923255/2/gate/nova-grenade-multinode/322833e/controller/logs/grenade.sh_log.txt
07:35:11 bauzas my only concern is that I'm not sure this is a transient issue
07:35:21 bauzas looks more a keystone regression to me
07:35:29 bauzas 2024-07-03 05:20:04.386 | testtools.matchers._impl.MismatchError: '56fc53e2e0724c0284ab80060ec55421' != '39aa3070a75e446fb6aa6d48bab74c58'
07:36:42 bauzas unless someone keystone-ed able to tell me this is something like a race condition in Keystone

Earlier   Later