| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-03-22 | |||
| 16:48:20 | artom | dansmith said someone is pulling out | |
| 16:48:31 | artom | (phrasing </archer>) | |
| 16:48:39 | dansmith | periodic is probably not a big deal I would imagine | |
| 16:48:54 | dansmith | I think we're going to need to trim down nova's per-patch jobs too, as it's getting pretty heavy | |
| 16:49:40 | bauzas | yeah, I don't think this is a big thing if we add a periodic | |
| 16:49:58 | gmann | yeah, and periodic also we can see if daily or weekly? | |
| 16:49:58 | bauzas | dansmith: adding a PTG topic about it fwiw | |
| 16:50:56 | bauzas | tbh, the only matter is how much we'll check its status and that will be weekly (during the team meeting) | |
| 16:51:04 | gmann | bauzas: artom along with periodic, add in experimental pipeline too for manual trigger. that helps to avoid adding it in check/gate pipeline if anyone want to run maually | |
| 16:51:38 | artom | bauzas, yep, no point in making it daily if we're only checking the status weekly | |
| 16:51:44 | artom | gmann, ack, can do | |
| 16:51:45 | gmann | +1 | |
| 16:51:57 | dansmith | yeah daily seems excessive | |
| 16:53:07 | bauzas | artom: update this change with the weekly period time and mention in the commit msg we'll need to verify it during weekly meetings | |
| 16:53:43 | bauzas | look at the placement ones | |
| 16:53:50 | artom | Oh yeah! | |
| 16:53:56 | gmann | artom: https://github.com/openstack/placement/blob/master/.zuul.yaml#L64 | |
| 16:54:00 | gmann | yeah | |
| 16:54:15 | bauzas | this is another pipeline IIRC | |
| 16:54:15 | artom | Hah, that was easy | |
| 16:54:20 | sean-k-mooney | by the way i think weekly jobs in general suit use better as we can review them in the weekly meeting | |
| 16:54:32 | sean-k-mooney | if we have a nightly we proably wont look at it every day | |
| 16:54:38 | bauzas | oh yeah | |
| 16:54:51 | bauzas | I just hope this meeting won't transform into some CI meeting | |
| 16:54:57 | chateaulav | artom: nova zuul has an example of weekly periodic now | |
| 16:55:02 | bauzas | if we start adding more periodics | |
| 16:55:22 | artom | I mean, feel free to nack the idea entirely :) | |
| 16:55:29 | sean-k-mooney | bauzas: well it should just be (are they green no we shoudl look at X after the meeting) | |
| 16:55:31 | artom | I'll obviously try to debate/convince you | |
| 16:55:37 | bauzas | artom: nah, I like the idea, I just want us to buy it | |
| 16:55:47 | artom | But if we think whitebox doesn't bring value to Nova CI, let's just not do it :) | |
| 16:56:04 | bauzas | we're approaching meeting's end time | |
| 16:56:14 | artom | End times are nigh | |
| 16:56:18 | bauzas | any other item to mention before we close ? | |
| 16:56:19 | sean-k-mooney | :) | |
| 16:56:33 | sean-k-mooney | ah i actully had two blueprints i wanted to raise | |
| 16:56:47 | sean-k-mooney | we defered updating the defaults for allcoation ratios | |
| 16:57:00 | bauzas | sean-k-mooney: oh I forgot to mention I changed Launchpad to reflect zed as the active series | |
| 16:57:16 | sean-k-mooney | shall we proceed with that or discuss at ptg | |
| 16:57:29 | sean-k-mooney | also kasyaps blueprint for usign the new libvirt apis | |
| 16:57:35 | bauzas | we're a bit short in time for reapproving specless bps by now | |
| 16:57:37 | sean-k-mooney | can we retarget both to zed | |
| 16:57:45 | sean-k-mooney | ack | |
| 16:57:52 | sean-k-mooney | we can disucss it next week or at ptg | |
| 16:57:52 | bauzas | but we can look at them during next meeting | |
| 16:58:03 | bauzas | well, Zed is open | |
| 16:58:13 | bauzas | I'm OK with approving things by now | |
| 16:58:19 | bauzas | and the specs repo is ready | |
| 16:58:50 | bauzas | sean-k-mooney: just propose your two blueprints for the next meeting so we'll reapprove them (unless concerns of course) | |
| 16:59:04 | sean-k-mooney | ack | |
| 16:59:29 | bauzas | fwiw, I leave the non-implemented blueprints in Deferred state | |
| 17:00:01 | bauzas | once we start reapproving some, I'd change back their state | |
| 17:00:11 | bauzas | but anyway, we're on time | |
| 17:00:13 | bauzas | thanks all | |
| 17:00:16 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-03-22-16.00.log.html | |
| 17:00:16 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-03-22-16.00.txt | |
| 17:00:16 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-03-22-16.00.html | |
| 17:00:16 | opendevmeet | Meeting ended Tue Mar 22 17:00:16 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 17:00:16 | bauzas | #endmeeting | |
| 17:00:25 | elodilles | thanks bauzas o/ | |
| 17:00:34 | bauzas | was a productive meeting, after all | |
| 17:03:30 | EugenMayer | Is there any 'good way' to set the task-state of an instance that has been stuck in 'image backup' due to an issue in glance? so the field OS-EXT-STS:task_state is on "image_backup" | |
| 17:05:07 | EugenMayer | i see there is 'nova set --state' or 'nova reset-state' but both seeem to operate on the instance-power-state (OS-EXT-STS:power_state) or OS-EXT-STS:vm_state - but not the task-state | |
| 17:07:20 | zigo | sean-k-mooney: Yeah, this was an evacuate operation. | |
| 17:07:47 | dansmith | zigo: I thought you said live migrate? | |
| 17:07:50 | sean-k-mooney | zigo: ok the reason this breaks is for evacuation we only have 1 allocation in placemnt against both hosts | |
| 17:08:17 | sean-k-mooney | and since the souce host is over capstiy because you reduce the allocate ration the entire allcoation is considered invlaid | |
| 17:08:34 | sean-k-mooney | we disussed this at the ptg 1 or 2 ptgs ago | |
| 17:09:13 | sean-k-mooney | i cant recall if we said we should fix this after consumer types but i dont think we had a workaround other then tempoarly increase the allcoation ratio so its nolonger over commited | |
| 17:09:55 | dansmith | sean-k-mooney: we could also solve it the way we do for cold migration, which is hold the allocation on the source with the migration uuid right? | |
| 17:10:13 | sean-k-mooney | dansmith: yes we could that was on eof the options | |
| 17:10:50 | sean-k-mooney | im trying to find the launchpad bug | |
| 17:11:44 | bauzas | dansmith: sean-k-mooney: yeah, the Migration uuid for evacuate seems the better and cleaner approach | |
| 17:12:04 | sean-k-mooney | bauzas: that is what we were proposing doing | |
| 17:12:26 | sean-k-mooney | but i dont think anyone has worked on it since | |
| 17:13:39 | bauzas | :-) | |
| 17:13:47 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1943191 | |
| 17:13:49 | sean-k-mooney | that might be it | |
| 17:13:58 | EugenMayer | I'am looking on https://wiki.openstack.org/wiki/CrashUp/Recover_From_Nova_Uncontrolled_Operations to understand how to recover from the crashed task state 'image_backup' but i'am not sure how to actual act upon that. Should i use the nova api? | |
| 17:14:03 | sean-k-mooney | and https://bugs.launchpad.net/nova/+bug/1924123 | |
| 17:14:37 | bauzas | sean-k-mooney: some people expect bugs to be fixed automatically :) | |
| 17:14:55 | bauzas | we don't have yet AI bots smart enough to close the gaps | |
| 17:14:58 | sean-k-mooney | EugenMayer: the wiki is basicaly unmaintained | |
| 17:15:09 | EugenMayer | i see. Thank you | |
| 17:16:03 | sean-k-mooney | in the early days of openstack we used the wiki for sepc and project created docs(docs not by the docs team) | |
| 17:16:14 | EugenMayer | I'am really not sure hot to again recover from the failed task the proper way. The only way i yet know, which is huge is: reset the state, then restart the compute the vm is hosted so thee state is somewhat recovered | |
| 17:17:13 | sean-k-mooney | there is not way to recover form it really beyond that | |
| 17:17:29 | sean-k-mooney | we dont provide a api to allow taskt to be restarted | |
| 17:17:53 | dansmith | reset state and reboot the vm is what I'd try first, | |
| 17:18:00 | sean-k-mooney | yep same | |
| 17:18:03 | dansmith | not restarting the compute I'd hope | |
| 17:18:16 | sean-k-mooney | ya that normally shoudl not be required | |
| 17:18:25 | sean-k-mooney | i guess it woudl depend on why it failed | |
| 17:18:43 | dansmith | definitely not expected for anything like a glance thing | |
| 17:18:50 | EugenMayer | trying that. AFAIR i had to restart the entire compute last time. Anyway, trying that | |
| 17:19:17 | sean-k-mooney | do you recall way? | |
| 17:19:20 | sean-k-mooney | *why | |
| 17:19:23 | EugenMayer | dansmith well this happens the 4th time. A stuck glance image backup task leaves the task_state of the instance in a broken state | |
| 17:19:26 | dansmith | honestly restarting the compute shouldn't even do anything, AFAIK | |
| 17:19:59 | sean-k-mooney | i wonder if the main thread of the compute agent was blocked on an io operations | |