| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-06 | |||
| 16:11:10 | bauzas | #topic Release Planning | |
| 16:11:14 | bauzas | #link https://releases.openstack.org/caracal/schedule.html#nova | |
| 16:11:18 | bauzas | #info Caracal-3 (and feature freeze) milestone in 3 weeks | |
| 16:11:24 | bauzas | ^ | |
| 16:11:43 | bauzas | please try to review the series, please | |
| 16:11:48 | bauzas | #topic Review priorities | |
| 16:11:55 | bauzas | #link https://etherpad.opendev.org/p/nova-caracal-status | |
| 16:12:05 | bauzas | for that you can use the above etherpad ^ | |
| 16:12:15 | bauzas | #topic Stable Branches | |
| 16:12:21 | bauzas | elodilles: your time | |
| 16:12:31 | elodilles | \o/ | |
| 16:12:33 | elodilles | #info stable/yoga transitioned to Unmaintained (stable/yoga head tagged with yoga-eom, unmaintained/yoga branch cut from that tag, stable/yoga (will be) deleted (asap)) | |
| 16:12:42 | elodilles | #info patches on stable/yoga were abandoned -> cherry pick them to unmaintained/yoga if needed | |
| 16:12:49 | elodilles | #info NOTE: unmaintained/yoga CI is currently broken with many issues | |
| 16:13:07 | elodilles | it is mostly the unmaintained-core group(s) responsibility | |
| 16:13:17 | elodilles | which is mostly me, at the moment :D | |
| 16:13:34 | elodilles | #info stable gates don't seem blocked | |
| 16:13:42 | elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | |
| 16:13:53 | elodilles | that's all from me | |
| 16:14:42 | bauzas | thanks | |
| 16:14:47 | elodilles | np | |
| 16:15:02 | bauzas | #topic vmwareapi 3rd-party CI efforts Highlights | |
| 16:15:06 | bauzas | fwiesel: around ? | |
| 16:15:08 | fwiesel | #info Web Access [http://openstack-ci-logs.global.cloud.sap/ "Pipeline Overview"], [http://openstack-ci-logs.global.cloud.sap/openstack/nova/refs/changes/09/906809/5/index.html "Example Log Index"], [http://openstack-ci-logs.global.cloud.sap/openstack/nova/refs/changes/09/906809/5/tempest.html "Example Tempest"] | |
| 16:16:22 | fwiesel | Finally got a proper access working, and a very basic hand-cratfted index. granchild is working on a nicer version, where you can pass links to logs with line-numbers | |
| 16:16:27 | bauzas | \o/ | |
| 16:17:03 | fwiesel | As you can see, if you cannot boot with a nova root disk, the tests are mostly red. I am now trying to figure out, why that happens. | |
| 16:17:22 | fwiesel | I haven't raised a bug yet, because I am not sure, that it is an issue in our setup. | |
| 16:17:56 | bauzas | ok | |
| 16:18:05 | fwiesel | That's from my side. | |
| 16:18:11 | fwiesel | Any questions? | |
| 16:18:58 | bauzas | nope from me | |
| 16:19:19 | fwiesel | Then back to you. | |
| 16:19:42 | bauzas | ok, last topic | |
| 16:19:48 | bauzas | #topic Open discussion | |
| 16:19:52 | bauzas | none in the agenda | |
| 16:19:56 | bauzas | any question ? | |
| 16:21:00 | bauzas | looks not | |
| 16:21:05 | bauzas | ok, thanks all | |
| 16:21:07 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2024/nova.2024-02-06-16.00.log.html | |
| 16:21:07 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2024/nova.2024-02-06-16.00.txt | |
| 16:21:07 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2024/nova.2024-02-06-16.00.html | |
| 16:21:07 | opendevmeet | Meeting ended Tue Feb 6 16:21:07 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:21:07 | bauzas | #endmeeting | |
| 16:21:21 | fwiesel | Thanks a lot. | |
| 16:22:05 | elodilles | thanks o/ | |
| 16:48:08 | opendevreview | Merged openstack/nova master: reno: Update master for unmaintained/yoga https://review.opendev.org/c/openstack/nova/+/908040 | |
| 16:48:11 | opendevreview | Merged openstack/placement master: reno: Update master for unmaintained/yoga https://review.opendev.org/c/openstack/placement/+/908066 | |
| 17:32:19 | melwitt | bauzas: looking at the nova-ovs-hybrid-plug job, not a fail reason I was expecting "keystoneauth1.exceptions.catalog.EndpointNotFound: publicURL endpoint for volumev3 service not found". I'll try to find what/how that's happening | |
| 17:32:42 | bauzas | cool | |
| 17:36:19 | melwitt | lol, I guess this would be why https://github.com/openstack/nova/commit/c486cc89dc11c2698cbf19fe8b2e477666331dee | |
| 17:37:47 | melwitt | cinder was removed from the job but a boot from volume test is still running | |
| 17:41:38 | melwitt | ah, it's not a test per se, but during resource setup it tries to create a bfv instance | |
| 17:49:17 | tkajinam | I wonder if anyone has ever seen this unicode error https://paste.opendev.org/show/byXf2AJyRTKB3Ck8VE9L/ | |
| 17:52:23 | tkajinam | maybe something has changed in libvirt and is causing that wired result | |
| 17:57:49 | opendevreview | melanie witt proposed openstack/nova master: run-evacuate-hook: Check cinder before creating BFV server https://review.opendev.org/c/openstack/nova/+/908182 | |
| 18:02:25 | melwitt | bauzas: hopefully ^ will work. it's weird though bc cinder was disabled in the job months ago yet it only started failing recently? | |
| 18:03:16 | sean-k-mooney | tkajinam: yep its an upstream libvirt bug | |
| 18:03:22 | sean-k-mooney | its fixed in libvirt 10 | |
| 18:03:46 | sean-k-mooney | tkajinam: we are trying to get that fixed in cetos9stream | |
| 18:04:02 | tkajinam | sean-k-mooney, ah, ok. if you are aware of it then that's nice. | |
| 18:04:07 | sean-k-mooney | tkajinam: one workaround would be to enabel the virt-preview corp repo | |
| 18:04:22 | tkajinam | I started hitting this a few hours ago, since libvirt 9.10.0 was released in c9s mirror | |
| 18:04:40 | tkajinam | in CI | |
| 18:05:15 | sean-k-mooney | we started seeing it last week in other cis that dont have a mirror | |
| 18:06:00 | melwitt | for the nova-lvm failures, so far I'm seeing a lot of glanceclient.exc.HTTPBadGateway: HTTP 502 | |
| 18:06:01 | sean-k-mooney | so i guess the other workaround is to clamp to libvirt <9.10.0 | |
| 18:06:16 | tkajinam | yeah. there may be some delay between upstream c9s mirror and opendev infra mirror | |
| 18:06:28 | opendevreview | Merged openstack/python-novaclient master: Update python classifier in setup.cfg https://review.opendev.org/c/openstack/python-novaclient/+/904617 | |
| 18:06:55 | tkajinam | I'll try that pin. afaik libvirt is not installed in the default image so pinning it may be easy | |
| 18:06:58 | tkajinam | sean-k-mooney, thanks ! | |
| 20:08:04 | melwitt | sean-k-mooney: this is a fix for the nova-ovs-hybrid-plug job, which is at a 100% fail rn afaict https://review.opendev.org/c/openstack/nova/+/908182 | |
| 20:14:09 | sean-k-mooney | melwitt: cool ill have a look quickly. iw as not really aware we did evacuate testing in that job | |
| 20:14:37 | sean-k-mooney | i tought i turned off cinder and swift in that job | |
| 20:14:42 | melwitt | yeah I didn't really know either | |
| 20:15:20 | sean-k-mooney | well it passed so ok +2w | |
| 20:15:53 | melwitt | yes cinder was disabled but the resource setup phase of the evacuate test stuff boots a bfv server, which fails bc no cinder endpoint. but how was it ever working before that? I don't see the reason | |
| 20:16:09 | sean-k-mooney | that job is specificaly for testing ml2/ovs so we tried to keep it small and not run things that were unrelated to the networking backend | |
| 20:16:16 | melwitt | I saw that the cinder disable patch was from september last year so I don't get it | |
| 20:16:45 | sean-k-mooney | did we recently add the evacuate hook or something | |
| 20:18:02 | melwitt | the evacuate hook has been around for a long time. and the patch that disabled cinder actually addressed some things related to evacuate https://review.opendev.org/c/openstack/nova/+/893540 | |
| 20:18:31 | sean-k-mooney | https://github.com/openstack/nova/commit/c486cc89dc11c2698cbf19fe8b2e477666331dee#diff-a27241a0d23e0378e836ec8459debe520a3ac0a18b8dbd7acc6e4cefc15ee75c | |
| 20:18:47 | melwitt | it's just it didn't guard around creation of a bfv server in setup_evacuate_resources.sh so I dunno why it didn't fail at that time, I would have thought it should have | |
| 20:18:53 | sean-k-mooney | so in the change we did to disabel cinder we tried to make it skip the bfv tests | |
| 20:19:04 | melwitt | right | |
| 20:19:49 | sean-k-mooney | ya so that should have alwasy just not work or passed | |
| 20:19:59 | sean-k-mooney | there must be somethign else happening but ok | |
| 20:22:33 | sean-k-mooney | i mean you directly modifed the failign code here but --poll on nova client is the same as --wait right | |
| 20:22:36 | sean-k-mooney | https://github.com/openstack/nova/commit/e96ac439d314358b52a17333fb13bd15a8a8e2c0 | |
| 20:23:23 | sean-k-mooney | actuly your change also merged before the cidner disabel one | |
| 20:23:48 | sean-k-mooney | melwitt: while you are around did you have time to respin the ephmeral encyption series | |
| 20:24:01 | sean-k-mooney | if so ill pull it down tomorrow and try and review it again | |
| 20:26:30 | melwitt | sean-k-mooney: I did. I'm going to respin once more today because there's something wonky with the encrypted backing files patch but it should be ready to pull down tomorrow | |
| 20:27:33 | melwitt | yeah --poll is the same as --wait in osc | |
| 20:28:23 | sean-k-mooney | cool | |
| 20:41:58 | melwitt | looking at a random passing nova-ovs-hybrid-plug run from a few days ago, it was failing to create the bfv server but it wasn't fatal for some reason https://zuul.opendev.org/t/openstack/build/6156ed6d9d1644d492300b2c33716ce2/log/job-output.txt#29530-29534 | |
| 20:44:18 | sean-k-mooney | odd im not sure why that woudl not have consitent behavior | |
| 20:44:59 | sean-k-mooney | maybe osc changerd its return code | |
| 20:45:14 | sean-k-mooney | i.e. in the past it might have returned 0 even if wait had an error | |
| 20:49:59 | melwitt | oh hm yeah | |