| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-09-07 | |||
| 16:32:51 | gibi | we have but we can have multiple unfinished migration object with type evac | |
| 16:33:06 | gibi | having a migration object does not mean the action of that migration can progress | |
| 16:33:18 | bauzas | hmmmm | |
| 16:33:21 | gibi | maybe the dest or the source of the migration is dead and you want to evac :) | |
| 16:33:53 | gibi | so a perfect solution is impossible I think | |
| 16:33:58 | gibi | we need to compromise some requirements | |
| 16:34:15 | tkajinam | the most safe approach might be add new api microversion to require task_state=None and require operators to do reset-state | |
| 16:34:21 | tkajinam | though I know that's not something operators may like | |
| 16:34:45 | gibi | I will never ask an operator in our do to do a reset-state | |
| 16:34:58 | gibi | that is like asking for a bigger problem because you had a small one :) | |
| 16:35:19 | gibi | s/in our do/ in our doc/ | |
| 16:35:33 | tkajinam | hmm ok. I agree | |
| 16:36:01 | gibi | I mean I spent so many time in customer cases where the deployer used reset-state and created a big mess | |
| 16:36:12 | gibi | maybe, a smarter reset-state... | |
| 16:36:23 | tkajinam | I know :-) | |
| 16:36:24 | gibi | but that is probably the same problem of deciding what action can be reset safely | |
| 16:36:50 | gibi | the same task_state can be ignored in evac precondition as well | |
| 16:37:20 | bauzas | are we already checking in the API evacuation API whether an evacuation is already run ? | |
| 16:38:22 | gibi | we checked in the past by looking at the task_state, but we removed that check effectively when we started allowing to evacuate from task_state=powering_off to support the above use cae | |
| 16:38:26 | gibi | case | |
| 16:39:54 | bauzas | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L5747 | |
| 16:40:08 | gibi | btw task_state=power_off can happen *during* an evac, as we started evacuating to STOPed state in microvesrion 2.95 | |
| 16:40:32 | bauzas | here, we ask for a task state to be None, right? | |
| 16:40:59 | gibi | bauzas: that task_state=None means do not check task_state, the code that expects an empty task_state is expected_task_state=(None,) | |
| 16:41:03 | gibi | or [None] | |
| 16:41:05 | bauzas | sorry, yeah | |
| 16:41:16 | bauzas | so we could say that we expect no task state | |
| 16:41:31 | gibi | no we cannot as that breaks the evac from task_state=powering_off case | |
| 16:41:33 | bauzas | but if we have an exception, we could look at the task state | |
| 16:41:36 | gibi | we support since 2022 | |
| 16:42:14 | bauzas | I mean, if we get a task state here, we can look whether this is due to an already existing rebuild task | |
| 16:42:23 | gibi | the problem is basically what task_state is allowed and what does not | |
| 16:42:24 | bauzas | if so, we can say "mmmh, nope" | |
| 16:43:02 | gibi | I don't have a list of allowable task_states | |
| 16:43:19 | gibi | I don't even have a list of definietly rejectable task_states | |
| 16:43:42 | gibi | as rebuild* task_state could mean an ongoing evac, or a failed evac that you want to retry | |
| 16:43:55 | gibi | where failed, is failed do the dest compute is dead | |
| 16:44:07 | gibi | s/do the/ do to/ | |
| 16:44:17 | gibi | s/do the/ due to/ | |
| 16:44:59 | bauzas | oh right the problem is not only when concurrent evacuation requests, this is also about ensuring we can ask for evac again if it fails | |
| 16:45:34 | bauzas | but if the evac failed, then the instance is not in ACTIVE vm state, right ? | |
| 16:45:40 | bauzas | I have to look at the code | |
| 16:46:02 | bauzas | I don't think we move the instance back to ACTIVE if we fail on evac | |
| 16:46:14 | bauzas | (compared to migrations for example) | |
| 16:46:29 | gibi | or write a book about distributes systems :) | |
| 16:46:29 | gibi | I guess I can propose PTG topic :) | |
| 16:46:58 | tkajinam | if evacuation "fails" normally it may be easy though if compute node dies in the middle then evacuation may get stuck | |
| 16:47:07 | tkajinam | can we check state of the dest compute node to determine whether the current evac can progress ? | |
| 16:47:40 | opendevreview | Ashish Gupta proposed openstack/nova master: Fix init_host crash migration test under native threading https://review.opendev.org/c/openstack/nova/+/1001369 | |
| 16:48:04 | gibi | bauzas: evac is possible from ERROR state :) | |
| 16:48:23 | bauzas | doh | |
| 16:48:39 | gibi | tkajinam: the dest compute is not well defined unfortunately | |
| 16:48:55 | bauzas | gibi: if you write a book about distributed systems, you wouldn't be the first :) | |
| 16:49:18 | gibi | there is instance.host that evacuate sets pretty late. There is the migration object dest host but we can have multiple unfinished evacuation migration object with different dests | |
| 16:49:25 | bauzas | I guess this sounds a PTG discussion indeed | |
| 16:49:33 | tkajinam | yeah | |
| 16:49:44 | gibi | bauzas: mine will solve it! | |
| 16:49:45 | gibi | :) | |
| 16:49:52 | bauzas | hah then I can review | |
| 16:50:07 | gibi | you will get a dedicate pre-release copy of it for sure | |
| 16:50:07 | bauzas | provided you're not proposing a state machine :) | |
| 16:50:25 | gibi | "a" state machine that is short sighted :) | |
| 16:50:51 | bauzas | if someone says "etcd will solve that", I swear I will find you | |
| 16:50:54 | gibi | OK I think I will propose a ptg topic | |
| 16:51:06 | Uggla | etcd will solve that :) | |
| 16:51:12 | bauzas | I'll find you | |
| 16:51:17 | Uggla | ;) | |
| 16:51:17 | gibi | there are "better" (mean newer) trials than etcd :) | |
| 16:51:17 | bauzas | and I know your home | |
| 16:51:45 | gibi | anyhow I think I'm done here. mic drop | |
| 16:51:52 | bauzas | gibi: so, are you asking for reviews about your WIP or do we punt that to PTG? | |
| 16:52:21 | gibi | bauzas: feel free to read the commit message of my WIP and my last comment on the bug | |
| 16:52:25 | bauzas | it will require a substantial amount of context loading, but that sounds a nasty bug tho | |
| 16:52:39 | gibi | but we won't have solution on RC1 week | |
| 16:52:42 | bauzas | gibi: OK'll try | |
| 16:52:47 | Uggla | To my mind that sounds complex enough to discuss to the PTG. | |
| 16:52:48 | gibi | so we can continue after H release | |
| 16:52:57 | gibi | probably on the PTG (depending of downstream push) | |
| 16:53:13 | Uggla | ok | |
| 16:53:28 | bauzas | my concern is that a PTG discussion will probably require some prerequisite effort for context loading | |
| 16:53:45 | gibi | I will create more context if this will end up being a PTG topic | |
| 16:53:53 | bauzas | cool | |
| 16:53:55 | gibi | I cannot help loading it though | |
| 16:54:07 | Uggla | anything else to discuss in the last 5mn, if not I think we are done. | |
| 16:54:14 | gibi | I don't yet have a remote exploit in your head | |
| 16:54:35 | bauzas | well, I think I know how we manage our VM and task states, but I'm more concerned by the quirks we have in code (all the nasty assumptions we have as code conditionals) | |
| 16:54:37 | gibi | (sssh do not tell them) | |
| 16:55:12 | bauzas | I could write a book about all nova API preconditions | |
| 16:55:49 | bauzas | (that one would probably be new compared to some random book about distributed systems :D ) | |
| 16:56:50 | Uggla | ok so closing now. | |
| 16:56:55 | Uggla | Thanks for joining this meeting. Have a nice day/evening. | |
| 16:57:02 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2026/nova.2026-09-07-16.00.log.html | |
| 16:57:02 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2026/nova.2026-09-07-16.00.txt | |
| 16:57:02 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2026/nova.2026-09-07-16.00.html | |
| 16:57:02 | opendevmeet | Meeting ended Mon Sep 7 16:57:02 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:57:02 | Uggla | #endmeeting | |
| 16:57:17 | bauzas | interesting talks, thanks | |
| 16:58:02 | gibi | thanks folks | |
| 16:58:09 | gibi | I'm dropping for today | |
| 16:59:12 | tkajinam | thanks | |