| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-07-03 | |||
| 18:09:04 | dansmith | +2 | |
| 18:09:14 | gmann | abhishek_: +w another one also, one comment for releasenotes but can be done in follow up https://review.opendev.org/c/openstack/tempest/+/923357/3/tempest/config.py#648 | |
| 18:09:15 | fungi | and both have approvals now, so doing | |
| 18:09:36 | gmann | ++ | |
| 18:09:41 | dansmith | thank you gmann | |
| 18:10:28 | gmann | np! sorry for login late. my toddler took a lot of time for his breakfast today :) | |
| 18:10:50 | fungi | okay all done and dequeued glance's 923433,2 at the same time | |
| 18:11:08 | abhishek_ | gmann: ack, thank you! | |
| 18:11:20 | dansmith | fungi: do it do it! | |
| 18:11:21 | abhishek_ | fungi ++ | |
| 18:11:48 | dansmith | YASS | |
| 18:12:05 | abhishek_ | :D | |
| 18:12:07 | fungi | count on me for all your fanservice needs | |
| 18:12:14 | dansmith | heh | |
| 18:19:11 | sean-k-mooney | dansmith: looks like gmann +w'd it | |
| 18:19:30 | sean-k-mooney | so i guess we want to wait for https://review.opendev.org/c/openstack/tempest/+/923352 to merge | |
| 18:19:36 | dansmith | sean-k-mooney: yes, it just unfolded as soon as I brought it up :D | |
| 18:19:59 | dansmith | sorry for the noise | |
| 20:36:04 | fungi | and both merged about 35 minutes ago now | |
| 20:51:13 | dansmith | ++ | |
| 21:23:12 | fungi | looking into options to less-disruptively speed merging for the remaining security patches... are there any other changes not explicitly mentioned in the ossa which i should avoid sending to the back of the line? | |
| 21:25:02 | fungi | going to start fiddling with some surgical queue reordering in about an hour once i get my dinner down | |
| 21:27:24 | melwitt | fungi: https://review.opendev.org/c/openstack/nova/+/923344 is a backport that some glance CVE patches depend on | |
| 21:27:54 | melwitt | (that I don't see mentioned in the ossa) | |
| 21:56:30 | sean-k-mooney | melwitt: that not part of the cve fix really | |
| 21:56:38 | sean-k-mooney | melwitt: thats just a ci change | |
| 21:57:29 | melwitt | sean-k-mooney: I know, but it's my understanding that it's required for some of the glance CVE backports. so it probably should be avoided to send to the back of the line | |
| 22:02:13 | clarkb | the entire gate just reset I think | |
| 22:02:20 | clarkb | so now is a good time to reorder cc fungi | |
| 22:02:46 | sean-k-mooney | melwitt: ah good point | |
| 22:11:54 | fungi | just got back | |
| 22:12:52 | fungi | assembling the list now | |
| 22:23:09 | fungi | mmm, this is going to be tricky because i have to pass in both the project name and the change,revision for each change i evict and re-add, so a simple loop won't suffice | |
| 22:23:12 | dansmith | yeah, that one is required for the glance on the same branch | |
| 22:25:21 | dansmith | poor glance hasn't merged anything yet, but the head change is in gate right now | |
| 22:26:19 | fungi | should https://review.opendev.org/c/openstack/glance/+/923433 be preserved too? it switches a job to non-voting | |
| 22:26:39 | dansmith | fungi: this got kicked out of gate but should be able to right back in: https://review.opendev.org/c/openstack/nova/+/923258/3 | |
| 22:26:43 | dansmith | and the parent of it is in gate now I think | |
| 22:27:02 | fungi | i can insert it between the dequeue and reenqueue steps, sure | |
| 22:27:30 | dansmith | fungi: I think abhi still wanted that mostly to insulate them from job timeouts, but it's not strictly required and I think it'd be better if we don't drop all that testing now that tempest is merged | |
| 22:27:43 | fungi | wfm | |
| 22:27:43 | dansmith | so you'd have my blessing to kick that out and make space, or I can if you want | |
| 22:28:27 | fungi | adding it to the list of stuff i'll move to the back of the line | |
| 22:28:32 | dansmith | ++ | |
| 22:36:34 | fungi | okay, surgery complete | |
| 22:37:10 | fungi | though in retrospect, as none of the changes at the top were being kept, it ended up being the same as if i'd done a promote of the ossa changes i guess | |
| 22:45:50 | fungi | i guess i expected there to be more changes from the ossa already approved | |
| 22:46:11 | dansmith | meaning the stable ones? | |
| 22:46:18 | fungi | turned out there were only 5 in the gate (plus the one that was kicked out and re-added) | |
| 22:46:20 | dansmith | all the master ones should be approved I think | |
| 22:46:46 | dansmith | given how things have been going with the insane cross-dependencies I think we've been kinda trying to make sure they are merging in release order | |
| 22:47:01 | dansmith | but if you really want them all +Wd we can probable just do that | |
| 22:47:15 | dansmith | I've heard great things about melwitt's +W finger | |
| 22:47:15 | fungi | nah, it's okay | |
| 22:47:27 | melwitt | lol | |
| 22:47:43 | melwitt | yup, I'm really great at clicking buttons | |
| #openstack-nova - 2024-07-04 | |||
| 03:06:23 | opendevreview | Merged openstack/nova master: Additional qemu safety checking on base images https://review.opendev.org/c/openstack/nova/+/923257 | |
| 06:13:40 | opendevreview | Merged openstack/nova stable/2023.1: Fix disk_formats in ceph job tempest config https://review.opendev.org/c/openstack/nova/+/923344 | |
| 07:19:25 | opendevreview | Maxime Lubin proposed openstack/nova-specs master: USB over IP https://review.opendev.org/c/openstack/nova-specs/+/920687 | |
| 07:25:43 | frickler | more gate resets happening all the time. I'll regate https://review.opendev.org/c/openstack/nova/+/923258 now, but not promote in order to give the glance stack a chance first | |
| 07:28:37 | abhishekk | thank you! | |
| 07:32:37 | opendevreview | Maxime Lubin proposed openstack/nova-specs master: USB over IP https://review.opendev.org/c/openstack/nova-specs/+/920687 | |
| 07:49:25 | amorin | hey team, just wondering if there is any way to start nova-conductor when nova-placement is down? It seems nova-conductor refused to start if placement API is not up | |
| 07:49:49 | amorin | openstack.exceptions.NotSupported: The placement service for keystone.x.y.z xists but does not have any supported versions. | |
| 07:55:21 | frickler | amorin: what would be the use case for that? I don't think the conductor can do anything useful without placement? | |
| 07:56:58 | amorin | I mean, maybe it can start anyway, and try to reconnect later, at least it will start and answer to computes that need some data | |
| 08:06:52 | frickler | amorin: hmm, maybe, but then let me ask a different question: in what scenario would you expect your placement service to be down for an extended period? or would just retrying a couple of times at startup be enough? | |
| 08:19:09 | amorin | agree that this scenario of placement down is wrong | |
| 08:20:44 | amorin | I am just surprised that this is the behavior of nova-conductor. | |
| 08:27:03 | amorin | my good old stein openstack was not complaining about this :) | |
| 08:55:15 | frickler | did that even have the split placement service yet? ;) | |
| 08:57:00 | amorin | yes, placement was splitted (at least in our deployment) | |
| 09:01:40 | gibi | amorin: I think by https://github.com/openstack/nova/commit/232684b44022f1bc4d72b07045900780de456e63 | |
| 09:01:49 | gibi | it shoud not happen any more | |
| 09:02:02 | bauzas | this is indeed an expected behaviour | |
| 09:11:35 | amorin | perfect! | |
| 09:12:08 | amorin | will try cherry-picking that to my antelope openstack then | |
| 09:12:21 | amorin | but wait, it should be there | |
| 09:15:44 | amorin | the exeption raised is the following: exceptions.NotSupported | |
| 09:16:17 | amorin | which is not in the first except block, so it's conesidered as a fatal one | |
| 09:16:47 | amorin | my service is stop and returning http 503 | |
| 09:26:17 | gibi | amorin: do you have placement in openstack endpoint list? | |
| 09:32:55 | frickler | amorin: that sounds like the keystone lib might be mishandling that error | |
| 09:35:12 | amorin | yes gibi, it's all set correctly, as soon as I start placement, everything works fine :) | |
| 09:37:01 | frickler | frickler: can you try to run something like "openstack --debug resource provider list" while placement is down and show the output on paste.opendev.org? (and possibly let's move this to #openstack-sdks) | |
| 09:37:09 | frickler | ehm amorin ^^ | |
| 09:51:05 | sean-k-mooney | amorin: this is a similar but slightly diffent issue | |
| 09:51:43 | sean-k-mooney | the orginal fix was to allow it to start when there was a keyston outaage not placment actully down | |
| 09:51:58 | sean-k-mooney | i think we have a diffent patch to make loading the placement client lazy | |
| 09:52:15 | sean-k-mooney | so that provided you dont make an api call that need it to talk to palcment it wont try | |
| 09:52:38 | sean-k-mooney | we worked on that for the schduler too but i dont recall whicih serviecs got that change and whihc did not | |
| 09:53:36 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/878238 that was the scheduler change | |
| 09:56:20 | sean-k-mooney | i guess it thet same apprch https://github.com/openstack/nova/commit/232684b44022f1bc4d72b07045900780de456e63 make the report clint a property and lazy loads it too | |
| 09:58:28 | sean-k-mooney | my guess is that a "503 Service Unavailable" responce does not count as ConnectionFailure and we just need to add that to the first expection block to cover your edgecase | |
| 10:10:53 | frickler | seems there is another different set of error classes for that, like in this case keystoneauth1.exceptions.http.ServiceUnavailable | |
| 10:15:01 | sean-k-mooney | ya so we just need to add them | |
| 10:15:23 | sean-k-mooney | with that said nova does expect placment ot be aviabel when wee start up we just added this lazy loading to make it more graceful | |
| 10:16:05 | sean-k-mooney | if placement is down for a protacted period of tiem we want to allow you to manage things that dont require interaction with placement but obvioulsy you wont be abel to boot new vms, evaucate them or move them in any way | |
| 10:16:36 | sean-k-mooney | start/stop/snapshot/attache port ectra is ok | |
| 10:17:02 | sean-k-mooney | but we would consdier the openstack to be in a degreated state and expect some operations ot fail | |