| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-05-11 | |||
| 10:51:51 | opendevreview | Merged openstack/nova master: Bump nova-ceph-multstore timeout https://review.opendev.org/c/openstack/nova/+/882890 | |
| 12:01:48 | gibi | sean-k-mooney: approved the os-vif qdisc patch | |
| 12:02:23 | sean-k-mooney | gibi: thanks for reviewing | |
| 12:08:01 | opendevreview | Merged openstack/os-vif master: remove focal based jobs https://review.opendev.org/c/openstack/os-vif/+/882755 | |
| 12:15:16 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Bump nova-ceph-multstore timeout https://review.opendev.org/c/openstack/nova/+/882784 | |
| 12:15:16 | bauzas | stable cores are needed for 2023.1 https://review.opendev.org/c/openstack/nova/+/882783 | |
| 12:16:12 | gibi | bauzas, dansmith, melwitt: The nova-ceph-mutlistore timeout bump is merged to master so I cherry-picked it to 2023.1 ^^ | |
| 12:16:37 | bauzas | gibi: done | |
| 12:16:51 | bauzas | gibi: sean-k-mooney: could you please look at https://review.opendev.org/c/openstack/nova/+/882783 ? | |
| 12:17:44 | sean-k-mooney | yep | |
| 12:18:39 | sean-k-mooney | elodilles: :) | |
| 12:18:49 | elodilles | done :) | |
| 12:18:53 | sean-k-mooney | bauzas: they are both now +2w'd | |
| 12:18:59 | gibi | \o/ | |
| 12:20:54 | sean-k-mooney | by the way i dont care that w is a letter can cant be past tense but +2w'd is definelty a thing :) and the 'd gets that point across | |
| 12:21:46 | sean-k-mooney | english sometime allows you to convay info in a way that would make tech writers cry | |
| 12:22:40 | sean-k-mooney | and im ok with that, we all know i mainly comunicate in seanspeak anyway :) | |
| 12:22:41 | elodilles | yepp, i also use EOL'd +2'd +W'd o:) | |
| 12:25:52 | elodilles | i don't remember where i saw that 1st time, but it can be understood, so i started to use that myself too | |
| 12:27:46 | sean-k-mooney | i dont think any one has ever objected but from a grammer rules point of view english allows you to syntasies past tense verbs this way. i just know my english teach would have been unhappy | |
| 12:28:22 | sean-k-mooney | with makeing an acronym both be a verb and have tence | |
| 12:28:50 | sean-k-mooney | ' is a very powerful thing :) | |
| 12:30:29 | elodilles | :) | |
| 12:43:31 | Uggla | @bauzas, gibi, if you can have a look at those small patches https://review.opendev.org/c/openstack/tempest/+/882822/1, https://review.opendev.org/c/openstack/tempest/+/882823/2 as I said tuesday that may help to find the bug. | |
| 12:54:37 | ykarel | sean-k-mooney, bauzas can you please revisit https://review.opendev.org/c/openstack/nova/+/868419 when get a chance | |
| 12:56:17 | opendevreview | Merged openstack/nova stable/2023.1: Enable use of service user token with admin context https://review.opendev.org/c/openstack/nova/+/882859 | |
| 13:23:36 | dansmith | gibi: thanks! | |
| 13:28:40 | opendevreview | Amit Uniyal proposed openstack/nova stable/wallaby: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/882939 | |
| 13:29:57 | bauzas | dansmith: thanks btw. for having +2d on my way | |
| 13:30:20 | bauzas | upstream CVE bugfixes are already merged by master and 2023.1 \o/ | |
| 13:32:06 | dansmith | bauzas: no problem | |
| 13:42:10 | opendevreview | Merged openstack/os-vif master: set default qos policy https://review.opendev.org/c/openstack/os-vif/+/881751 | |
| 13:52:53 | gibi | Uggla: done. I don't have +2 rights in tempest but those patches looks good | |
| 13:53:52 | Uggla | gibi, cool thx. | |
| 13:56:30 | dansmith | bauzas: gibi so just an update, based on my opensearch digging, I don't think we have seen any volume detach failures in the last week with the exception of cases where either ceph oomed or the guest had a kernel panic | |
| 13:56:50 | dansmith | not definitive for sure, but based on previous behavior, I think that's massively better | |
| 14:21:24 | opendevreview | Merged openstack/nova stable/2023.1: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882783 | |
| 14:25:11 | bauzas | dansmith: bravo to you | |
| 14:25:34 | dansmith | also, the cve backport just failed on the backport validator | |
| 14:25:42 | dansmith | maybe just github not updated yet? | |
| 14:25:45 | opendevreview | Elod Illes proposed openstack/nova master: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882956 | |
| 14:26:52 | elodilles | bauzas dansmith : we need this fix as new branch naming broke our validator :S ^^^ | |
| 14:27:05 | dansmith | ah | |
| 14:27:42 | dansmith | +2 | |
| 14:27:50 | elodilles | thx | |
| 14:27:53 | dansmith | elodilles: nice job thanks | |
| 14:28:28 | elodilles | i'll propose the backports if this is about to merge | |
| 14:28:34 | gibi | dansmith: nice results! | |
| 14:29:38 | dansmith | cinder is still running with validations disabled and so they're hitting all the ones we used to but we're not, so that's also a nice A/B comparison :) | |
| 14:48:22 | opendevreview | Oleksandr Klymenko proposed openstack/nova master: Host removed from AZ when service is manually disabled https://review.opendev.org/c/openstack/nova/+/882957 | |
| 15:05:01 | opendevreview | Elod Illes proposed openstack/nova stable/2023.1: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882964 | |
| 15:06:15 | opendevreview | Elod Illes proposed openstack/nova stable/zed: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882965 | |
| 15:07:26 | opendevreview | Elod Illes proposed openstack/nova stable/yoga: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882966 | |
| 15:08:38 | opendevreview | Elod Illes proposed openstack/nova stable/xena: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882967 | |
| 15:09:49 | opendevreview | Elod Illes proposed openstack/nova stable/wallaby: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882968 | |
| 15:11:00 | opendevreview | Elod Illes proposed openstack/nova stable/victoria: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882969 | |
| 15:12:10 | opendevreview | Elod Illes proposed openstack/nova stable/ussuri: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882970 | |
| 15:13:22 | opendevreview | Elod Illes proposed openstack/nova stable/train: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882971 | |
| 15:29:22 | opendevreview | Sylvain Bauza proposed openstack/nova stable/zed: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882786 | |
| 16:17:57 | bauzas | folks, I'm also saying it loudly, tomorrow I'll be working and looking at specs | |
| 16:52:21 | dansmith | melwitt: the backport checker fix is going to fail on functional, that db table race thing | |
| 16:54:43 | melwitt | argh | |
| 16:54:58 | dansmith | is there a bug open for that/ | |
| 16:55:56 | melwitt | yes, sec | |
| 16:56:10 | dansmith | I dunno why zuul hasn't kicked it out so I can recheck it yet | |
| 16:56:30 | melwitt | I believe it's this one https://bugs.launchpad.net/nova/+bug/1946339 | |
| 16:57:14 | dansmith | hmm, similar at least | |
| 16:57:18 | melwitt | gibi has done a lot of work to improve the situation but it's a gnarly issue | |
| 16:58:16 | dansmith | okay yeah that's the same | |
| 16:58:20 | JayF | Hey; I re-proposed the Ironic sharding spec for this cycle about 2 weeks ago. It's not gotten any reviews. If anyone can take a look I'd appreciate it: https://review.opendev.org/c/openstack/nova-specs/+/881643 | |
| 16:58:50 | melwitt | there is a change I think we could do that might help but I haven't proposed it yet bc it hadn't been happening very often for a long time there | |
| 16:59:14 | dansmith | melwitt: okay gibi just commented on the bug that we've seen an. uptick recently | |
| 16:59:37 | melwitt | yeah, I just saw that too | |
| 17:12:49 | dansmith | fungi: there's something that has completed all our jobs in gate that we need to recheck, but it's just sitting there in the queue and I'm not sure why | |
| 17:13:24 | dansmith | it's already identified as failed and out of the queue, but not.. uh, reporting or whatever | |
| 17:13:25 | clarkb | dansmith: because the changes ahead of it haven't finished | |
| 17:13:37 | clarkb | only the first thing in the queue can report | |
| 17:13:48 | clarkb | doing so removes it from the queue and then the next item can be processed | |
| 17:13:54 | dansmith | I thought if nothing in front could have caused the failure on that job it would come out immediately | |
| 17:14:04 | clarkb | zuul doesn't have that information so can't do that | |
| 17:14:14 | dansmith | hmm, okay | |
| 17:14:25 | clarkb | by putting things in the same queue you are asserting a failure in one may be caused by the other | |
| 17:14:31 | clarkb | and zuul is operating on that knowledge | |
| 17:14:37 | dansmith | okay | |
| 17:15:03 | dansmith | but it already shows it as failed out (meaning the fork in the line) so I thought that was it saying it knows the things behind it no longer depend | |
| 17:15:41 | clarkb | correct the things behind it no longer depend on it. But the things ahead of it may be where the actual bug is | |
| 17:16:02 | clarkb | in that case you want to evict the broken stuff ahead and restart the things ehind | |
| 17:16:18 | dansmith | so the green checks behind this are based on skipping it or with it applied? | |
| 17:16:38 | clarkb | the green checks behind are based on skipping the one that has failed | |
| 17:16:41 | dansmith | I assume with it applied and they'll restart if it decides it was legit to kick it out? | |
| 17:16:47 | dansmith | hmm okay | |
| 17:16:50 | clarkb | the unknown is the not yet completed jobs ahead of it | |
| 17:16:55 | dansmith | so if it doesn't get kicked out they restart? | |
| 17:16:57 | fungi | but they'll all be tested again from scratch if something else ahead of all of those fails a job | |
| 17:17:49 | clarkb | you have 5 changes, 6th is a failure, then X behind. Zuul does not know if the failure was caused by the 5 changes at the front so it does not completely evict the 6th until it processes the 5 ahead of it | |
| 17:17:50 | fungi | until all changes ahead of the failing change merge successfully, zuul can't be sure that there's something wrong with that change | |
| 17:18:18 | dansmith | it's too bad we can't mark a job as isolated or something, because this is only running nova unit tests, but it's held up as if it has the same dependencies as something with a tempest (which is why the queue needs to be shared) | |
| 17:18:28 | dansmith | obviously not a very common case | |
| 17:19:12 | dansmith | fungi: ack, the fork in the graph makes it look to me like it's already "out" but yeah okay | |
| 17:19:36 | dansmith | "out of consideration" I should say | |