| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-05-11 | |||
| 06:47:29 | gibi | both failed with http read timeout on various openstack APIs (cinder, neutron, nova) | |
| 06:48:05 | gibi | I think this is tracked here as a bug https://bugs.launchpad.net/tempest/+bug/1999893 | |
| 07:05:24 | bauzas | morning | |
| 07:05:45 | bauzas | gibi: catching up the world explosion after my yesterday PTO | |
| 07:29:12 | gibi | I don't have the full context as the CVE got public why I was away | |
| 07:29:20 | gibi | I see the fixes proposed so I'm trying to land them | |
| 07:29:25 | opendevreview | Amit Uniyal proposed openstack/nova stable/yoga: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/882920 | |
| 07:32:10 | gibi | s/why/while/ | |
| 07:32:16 | opendevreview | Sylvain Bauza proposed openstack/nova stable/2023.1: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882783 | |
| 07:33:07 | bauzas | gibi: np, just looking at gerrit reviews | |
| 07:33:14 | bauzas | gibi: do you need some explanations ? | |
| 07:33:43 | gibi | I read the CVE it is well documented so thanks I'm OK about the fixes we are pushing | |
| 07:34:00 | opendevreview | Amit Uniyal proposed openstack/nova stable/xena: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/882921 | |
| 09:47:34 | sean-k-mooney | bauzas: can you take a look at some os-vif changes for me | |
| 09:48:08 | sean-k-mooney | this one is trivial https://review.opendev.org/c/openstack/os-vif/+/882755 and this one is the one i really want to merge https://review.opendev.org/c/openstack/os-vif/+/881751 | |
| 09:48:33 | sean-k-mooney | or gibi or anyone else who is about | |
| 10:11:50 | bauzas | sean-k-mooney: fyk see my comment on https://review.opendev.org/c/openstack/os-vif/+/882755 | |
| 10:12:09 | bauzas | the TC will add back py38 in the Bobcat PTI | |
| 10:12:37 | bauzas | (but that's just a FYK, since Focal won't be accepted again) | |
| 10:25:46 | sean-k-mooney | 38 i htink is still tested in os-vif | |
| 10:25:59 | sean-k-mooney | i havent got around to updating that so cool i can leave that | |
| 10:26:31 | sean-k-mooney | we just use openstack-python3-jobs to contol that | |
| 10:27:01 | sean-k-mooney | i might add 3.11 like we did in nova seperatly but ill keep 3.8 testable | |
| 10:27:30 | sean-k-mooney | thanks for the reminder | |
| 10:32:52 | opendevreview | Amit Uniyal proposed openstack/nova stable/wallaby: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/882939 | |
| 10:51:51 | opendevreview | Merged openstack/nova master: Bump nova-ceph-multstore timeout https://review.opendev.org/c/openstack/nova/+/882890 | |
| 12:01:48 | gibi | sean-k-mooney: approved the os-vif qdisc patch | |
| 12:02:23 | sean-k-mooney | gibi: thanks for reviewing | |
| 12:08:01 | opendevreview | Merged openstack/os-vif master: remove focal based jobs https://review.opendev.org/c/openstack/os-vif/+/882755 | |
| 12:15:16 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Bump nova-ceph-multstore timeout https://review.opendev.org/c/openstack/nova/+/882784 | |
| 12:15:16 | bauzas | stable cores are needed for 2023.1 https://review.opendev.org/c/openstack/nova/+/882783 | |
| 12:16:12 | gibi | bauzas, dansmith, melwitt: The nova-ceph-mutlistore timeout bump is merged to master so I cherry-picked it to 2023.1 ^^ | |
| 12:16:37 | bauzas | gibi: done | |
| 12:16:51 | bauzas | gibi: sean-k-mooney: could you please look at https://review.opendev.org/c/openstack/nova/+/882783 ? | |
| 12:17:44 | sean-k-mooney | yep | |
| 12:18:39 | sean-k-mooney | elodilles: :) | |
| 12:18:49 | elodilles | done :) | |
| 12:18:53 | sean-k-mooney | bauzas: they are both now +2w'd | |
| 12:18:59 | gibi | \o/ | |
| 12:20:54 | sean-k-mooney | by the way i dont care that w is a letter can cant be past tense but +2w'd is definelty a thing :) and the 'd gets that point across | |
| 12:21:46 | sean-k-mooney | english sometime allows you to convay info in a way that would make tech writers cry | |
| 12:22:40 | sean-k-mooney | and im ok with that, we all know i mainly comunicate in seanspeak anyway :) | |
| 12:22:41 | elodilles | yepp, i also use EOL'd +2'd +W'd o:) | |
| 12:25:52 | elodilles | i don't remember where i saw that 1st time, but it can be understood, so i started to use that myself too | |
| 12:27:46 | sean-k-mooney | i dont think any one has ever objected but from a grammer rules point of view english allows you to syntasies past tense verbs this way. i just know my english teach would have been unhappy | |
| 12:28:22 | sean-k-mooney | with makeing an acronym both be a verb and have tence | |
| 12:28:50 | sean-k-mooney | ' is a very powerful thing :) | |
| 12:30:29 | elodilles | :) | |
| 12:43:31 | Uggla | @bauzas, gibi, if you can have a look at those small patches https://review.opendev.org/c/openstack/tempest/+/882822/1, https://review.opendev.org/c/openstack/tempest/+/882823/2 as I said tuesday that may help to find the bug. | |
| 12:54:37 | ykarel | sean-k-mooney, bauzas can you please revisit https://review.opendev.org/c/openstack/nova/+/868419 when get a chance | |
| 12:56:17 | opendevreview | Merged openstack/nova stable/2023.1: Enable use of service user token with admin context https://review.opendev.org/c/openstack/nova/+/882859 | |
| 13:23:36 | dansmith | gibi: thanks! | |
| 13:28:40 | opendevreview | Amit Uniyal proposed openstack/nova stable/wallaby: Have host look for CPU controller of cgroupsv2 location. https://review.opendev.org/c/openstack/nova/+/882939 | |
| 13:29:57 | bauzas | dansmith: thanks btw. for having +2d on my way | |
| 13:30:20 | bauzas | upstream CVE bugfixes are already merged by master and 2023.1 \o/ | |
| 13:32:06 | dansmith | bauzas: no problem | |
| 13:42:10 | opendevreview | Merged openstack/os-vif master: set default qos policy https://review.opendev.org/c/openstack/os-vif/+/881751 | |
| 13:52:53 | gibi | Uggla: done. I don't have +2 rights in tempest but those patches looks good | |
| 13:53:52 | Uggla | gibi, cool thx. | |
| 13:56:30 | dansmith | bauzas: gibi so just an update, based on my opensearch digging, I don't think we have seen any volume detach failures in the last week with the exception of cases where either ceph oomed or the guest had a kernel panic | |
| 13:56:50 | dansmith | not definitive for sure, but based on previous behavior, I think that's massively better | |
| 14:21:24 | opendevreview | Merged openstack/nova stable/2023.1: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882783 | |
| 14:25:11 | bauzas | dansmith: bravo to you | |
| 14:25:34 | dansmith | also, the cve backport just failed on the backport validator | |
| 14:25:42 | dansmith | maybe just github not updated yet? | |
| 14:25:45 | opendevreview | Elod Illes proposed openstack/nova master: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882956 | |
| 14:26:52 | elodilles | bauzas dansmith : we need this fix as new branch naming broke our validator :S ^^^ | |
| 14:27:05 | dansmith | ah | |
| 14:27:42 | dansmith | +2 | |
| 14:27:50 | elodilles | thx | |
| 14:27:53 | dansmith | elodilles: nice job thanks | |
| 14:28:28 | elodilles | i'll propose the backports if this is about to merge | |
| 14:28:34 | gibi | dansmith: nice results! | |
| 14:29:38 | dansmith | cinder is still running with validations disabled and so they're hitting all the ones we used to but we're not, so that's also a nice A/B comparison :) | |
| 14:48:22 | opendevreview | Oleksandr Klymenko proposed openstack/nova master: Host removed from AZ when service is manually disabled https://review.opendev.org/c/openstack/nova/+/882957 | |
| 15:05:01 | opendevreview | Elod Illes proposed openstack/nova stable/2023.1: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882964 | |
| 15:06:15 | opendevreview | Elod Illes proposed openstack/nova stable/zed: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882965 | |
| 15:07:26 | opendevreview | Elod Illes proposed openstack/nova stable/yoga: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882966 | |
| 15:08:38 | opendevreview | Elod Illes proposed openstack/nova stable/xena: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882967 | |
| 15:09:49 | opendevreview | Elod Illes proposed openstack/nova stable/wallaby: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882968 | |
| 15:11:00 | opendevreview | Elod Illes proposed openstack/nova stable/victoria: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882969 | |
| 15:12:10 | opendevreview | Elod Illes proposed openstack/nova stable/ussuri: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882970 | |
| 15:13:22 | opendevreview | Elod Illes proposed openstack/nova stable/train: CI: fix backport validator for new branch naming https://review.opendev.org/c/openstack/nova/+/882971 | |
| 15:29:22 | opendevreview | Sylvain Bauza proposed openstack/nova stable/zed: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882786 | |
| 16:17:57 | bauzas | folks, I'm also saying it loudly, tomorrow I'll be working and looking at specs | |
| 16:52:21 | dansmith | melwitt: the backport checker fix is going to fail on functional, that db table race thing | |
| 16:54:43 | melwitt | argh | |
| 16:54:58 | dansmith | is there a bug open for that/ | |
| 16:55:56 | melwitt | yes, sec | |
| 16:56:10 | dansmith | I dunno why zuul hasn't kicked it out so I can recheck it yet | |
| 16:56:30 | melwitt | I believe it's this one https://bugs.launchpad.net/nova/+bug/1946339 | |
| 16:57:14 | dansmith | hmm, similar at least | |
| 16:57:18 | melwitt | gibi has done a lot of work to improve the situation but it's a gnarly issue | |
| 16:58:16 | dansmith | okay yeah that's the same | |
| 16:58:20 | JayF | Hey; I re-proposed the Ironic sharding spec for this cycle about 2 weeks ago. It's not gotten any reviews. If anyone can take a look I'd appreciate it: https://review.opendev.org/c/openstack/nova-specs/+/881643 | |
| 16:58:50 | melwitt | there is a change I think we could do that might help but I haven't proposed it yet bc it hadn't been happening very often for a long time there | |
| 16:59:14 | dansmith | melwitt: okay gibi just commented on the bug that we've seen an. uptick recently | |
| 16:59:37 | melwitt | yeah, I just saw that too | |
| 17:12:49 | dansmith | fungi: there's something that has completed all our jobs in gate that we need to recheck, but it's just sitting there in the queue and I'm not sure why | |
| 17:13:24 | dansmith | it's already identified as failed and out of the queue, but not.. uh, reporting or whatever | |