| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-10-11 | |||
| 16:14:28 | bauzas | heh :) | |
| 16:14:44 | bauzas | as you see, we have a timeout with the centos9-fips job | |
| 16:15:20 | bauzas | but I dunno for how long it was having this timeout | |
| 16:16:20 | elodilles | https://zuul.openstack.org/builds?job_name=tempest-centos9-stream-fips&project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly&skip=0 | |
| 16:16:28 | elodilles | it seems always? | |
| 16:16:33 | bauzas | yes | |
| 16:17:53 | bauzas | I guess the owner of the job was adalee, right?. | |
| 16:18:40 | bauzas | this is bizarre, I don't see where the job run is timing out | |
| 16:20:01 | sean-k-mooney | well the default timeout i think is 2 hours | |
| 16:20:14 | sean-k-mooney | so it might just need a little longer sicne the fips job does a reboot | |
| 16:20:24 | bauzas | yeah but the job seems to have done | |
| 16:20:30 | bauzas | be* done | |
| 16:20:38 | bauzas | anyway, nothing urgent | |
| 16:20:53 | bauzas | this is just the fact we can't get the result | |
| 16:21:00 | bauzas | even if it looks tempest works fine | |
| 16:21:11 | sean-k-mooney | its posible it has addtional test in a post playbook | |
| 16:21:24 | bauzas | I don't know who wrote the job | |
| 16:21:34 | bauzas | but I'll try to find it | |
| 16:21:46 | bauzas | should be easy to find | |
| 16:22:24 | clarkb | https://zuul.openstack.org/build/5f6e6a2f65ee4a5e90754d07142fab9f/log/job-output.txt#23945-23946 shows where it is timing out | |
| 16:23:24 | bauzas | found who added it https://review.opendev.org/c/openstack/nova/+/831844 | |
| 16:23:54 | bauzas | thanks clarkb, will look | |
| 16:24:31 | sean-k-mooney | ok its tempest full | |
| 16:24:47 | sean-k-mooney | so its two tempest runs | |
| 16:25:05 | bauzas | | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tempest/playbooks/devstack-tempest.yaml@master] | |
| 16:25:09 | sean-k-mooney | the first one completed but the second that runs the slow test and senarior tests timed out | |
| 16:25:10 | bauzas | hmpf | |
| 16:25:24 | bauzas | yup | |
| 16:25:45 | sean-k-mooney | so ya it just need anther say 30 mins added to the tiemout to be safe | |
| 16:26:05 | sean-k-mooney | it also could have just been a slow node | |
| 16:26:10 | clarkb | and maybe double check logs to see that something didn't get stuck there due to fips | |
| 16:26:19 | clarkb | there is a bit of a time delta between the timeout and tempest reporting anything | |
| 16:26:38 | clarkb | oh its only 15 seconds nevermind | |
| 16:26:42 | sean-k-mooney | yep | |
| 16:26:47 | sean-k-mooney | there isnt really a break in the logs | |
| 16:26:53 | clarkb | I mathed wrong the first time | |
| 16:27:08 | bauzas | yes, this is just a slow test | |
| 16:27:42 | bauzas | but should we modify the timeout for that job to be larger ? | |
| 16:27:46 | sean-k-mooney | i dont know how much time the reboot adds for fips but 2 hours is proably borderlien | |
| 16:28:01 | bauzas | we can DNM this at least | |
| 16:28:16 | bauzas | and verify if adding more timeout time helps | |
| 16:28:17 | sean-k-mooney | i would add 30mins and monitor it and see roughly how long it takes over the next few weeks | |
| 16:28:23 | bauzas | right | |
| 16:28:46 | bauzas | I can propose a patch | |
| 16:28:51 | sean-k-mooney | ack | |
| 16:29:21 | bauzas | #action bauzas to add a 30min more timeout for the centos9-fips periodic job so we will see whether it fixes the timeout | |
| 16:29:27 | bauzas | moving on | |
| 16:29:38 | bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | |
| 16:29:51 | sean-k-mooney | bauzas: its here https://opendev.org/openstack/tempest/src/branch/master/zuul.d/integrated-gate.yaml#L325 just incase your wondering where to do that | |
| 16:31:19 | bauzas | ack, thanks but I found it | |
| 16:31:40 | bauzas | hence the DNM patch in nova | |
| 16:31:51 | bauzas | to test it | |
| 16:32:07 | bauzas | with experimental | |
| 16:32:10 | bauzas | anyway | |
| 16:32:36 | bauzas | nothing to tell more about blind rechecks, I haven't looked at the numbers | |
| 16:32:38 | bauzas | moving on | |
| 16:32:48 | bauzas | #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | |
| 16:32:53 | bauzas | #topic Release Planning | |
| 16:32:59 | bauzas | #link https://releases.openstack.org/zed/schedule.html | |
| 16:33:04 | bauzas | #info Zed GA was last week, kudos to the team. | |
| 16:33:10 | bauzas | we're officially done with Zed :) | |
| 16:33:21 | bauzas | so... | |
| 16:33:23 | bauzas | #link https://releases.openstack.org/antelope/schedule.html | |
| 16:33:28 | bauzas | #info Antelope-1 is planned in 5 weeks | |
| 16:33:38 | bauzas | nothing to tell it, the crux is for the next topic | |
| 16:33:50 | bauzas | #topic PTG planning | |
| 16:33:55 | bauzas | #link https://etherpad.opendev.org/p/nova-antelope-ptg Antelope PTG etherpad | |
| 16:34:00 | bauzas | #link https://ptg.opendev.org/ptg.html PTG schedule | |
| 16:34:09 | bauzas | #info Attempted schedule in the etherpad | |
| 16:34:21 | bauzas | yeah I did a bit of a topics scrambling | |
| 16:34:38 | bauzas | but honestly, we don't have a large agendat *yet* | |
| 16:35:04 | bauzas | I'll just move the nova-otherproject topics | |
| 16:35:29 | bauzas | for the moment, we have nova-neutron (Thur) and nova-ironic (reminder : *Monday*) | |
| 16:35:44 | bauzas | do people want to talk with cinder or other projects ? | |
| 16:35:52 | bauzas | I haven't got any ask from the cyborg project | |
| 16:36:05 | bauzas | or oslo or QA ? | |
| 16:36:25 | bauzas | or manila, or glance... :) | |
| 16:36:57 | gibi | nothing on my side | |
| 16:37:04 | sean-k-mooney | i think the ironic topic will fill any space we give it | |
| 16:37:14 | sean-k-mooney | but im not sure if we will be able to make progress remotely | |
| 16:37:57 | bauzas | hah | |
| 16:38:31 | bauzas | sean-k-mooney: as I already said a couple of times, I don't think we will find a solution | |
| 16:38:33 | bauzas | but, | |
| 16:38:55 | bauzas | at least I want our community to understand why we can't just modify an instance .host DB value | |
| 16:39:25 | sean-k-mooney | we also shoudl not have the hashring in the driver as we do today | |
| 16:39:35 | bauzas | if we're able to explain it even after all the comments dansmith and you did, well, it will be nice then | |
| 16:40:03 | sean-k-mooney | i look forward to your summary | |
| 16:40:16 | bauzas | anyway, I don't want to discuss this spec by now, we'll have time on Monday :) | |
| 16:40:31 | bauzas | sean-k-mooney: /o\ | |
| 16:40:46 | bauzas | but yeah, I'll summarize this session after the PTG :) | |
| 16:40:55 | bauzas | moving on | |
| 16:41:12 | bauzas | looks we can find other projects to discuss if we want during the PTG anyxway | |
| 16:41:32 | bauzas | as a reminder too, we could get pain points or feature requests from operators | |
| 16:41:51 | bauzas | #link https://etherpad.opendev.org/p/oct2022-ptg-operator-hour-nova Operator specific etherpad for our nova operator hours | |
| 16:41:59 | sean-k-mooney | i hope we do. even if they are ones we have gotten before | |
| 16:42:23 | bauzas | #topic Review priorities | |
| 16:42:29 | bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) | |
| 16:42:55 | bauzas | I need to add some changes I have as a review-prio | |
| 16:43:05 | bauzas | apart from this, nothing to tell | |
| 16:43:17 | bauzas | next topic ? | |