| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-09-12 | |||
| 08:58:22 | sahid | yes that makes sense, i understand now | |
| 08:58:38 | sean-k-mooney | but more because if one of the evacuation fails its kind of undefiend what the end result of the commnd is | |
| 08:59:01 | sahid | so back to the original use-case, does that would make sense to have evacuate with a flag to force the state? | |
| 08:59:12 | sean-k-mooney | it wont be one of (all evacuated or all still on orginal host) it will be a mix | |
| 08:59:29 | sean-k-mooney | sahid: i would say target state | |
| 08:59:33 | sean-k-mooney | rahter then force | |
| 08:59:46 | sean-k-mooney | that has been requested before at the last inperson ptg i think | |
| 09:00:02 | sean-k-mooney | i would not be apposed to a eveacuate to stopped option | |
| 09:00:12 | sean-k-mooney | im not sure that shelved makes sense | |
| 09:00:21 | sean-k-mooney | but started/stopped i can see | |
| 09:00:26 | gibi | I think target_state enum (AsBefore,Stopped) | |
| 09:00:28 | gibi | make sense | |
| 09:00:44 | gibi | AsBefore=NoChange | |
| 09:01:16 | gibi | can we evacuate a shelved instance? | |
| 09:01:26 | sean-k-mooney | on reset-sate while i would like to expand what it can do so that you can specify somehting other then aviable/error im not sure this is the right way to do this | |
| 09:01:31 | sean-k-mooney | gibi: no | |
| 09:01:38 | sean-k-mooney | gibi: because its not on a host | |
| 09:01:43 | gibi | OK, cool then :D | |
| 09:01:48 | gibi | I started worrying :) | |
| 09:02:01 | sean-k-mooney | im ment it would not make sense to evacuate to shelve_offloaded | |
| 09:02:21 | sean-k-mooney | we could allow shelve_offloading when its down instead | |
| 09:02:27 | sean-k-mooney | but its not really evacuate | |
| 09:02:47 | sean-k-mooney | evaucate is ment to move the vm form one host to another | |
| 09:03:06 | sean-k-mooney | where as shelve/unshleve is moving form on a host to not and vise versa | |
| 09:03:20 | gibi | yeah | |
| 09:03:40 | sean-k-mooney | its kind of a pendantic distinction but i dont quite consider them equal | |
| 09:03:47 | sean-k-mooney | you could argue it either way | |
| 09:04:15 | sean-k-mooney | so i would not be agaisn allowing stop ot work in a host dwonstate by the way | |
| 09:04:30 | sean-k-mooney | you woudl update the db and treat it kind of like local delete | |
| 09:04:43 | sean-k-mooney | when the compute agent comes back up it woudl reconsile the vm state | |
| 09:05:13 | sean-k-mooney | if you stoped it then evacuated that would solve sahid's case | |
| 09:05:20 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/854499 | |
| 09:05:21 | opendevreview | Amit Uniyal proposed openstack/nova master: [compute] always set instnace.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/791135 | |
| 09:05:43 | sahid | sean-k-mooney: yes it's also a possibility | |
| 09:06:56 | sean-k-mooney | the one thing to keep in mind i guess is that even if we allow stop | |
| 09:07:08 | sean-k-mooney | it doen not chnage the responsiblity for the admin | |
| 09:07:28 | sean-k-mooney | that is you are requried as an admin to ensure a host is fenced or all vms are stopped before you evacuate | |
| 09:08:00 | sean-k-mooney | if we allow stop in a down host state the admin still need to ensure it is stoped to prevent data currpption | |
| 09:08:17 | sean-k-mooney | but if they can then that woudl allwo them to evacuate without start the vm again | |
| 09:08:24 | gibi | this is why I would connect the stopping to the evacuation action, that way it is clear that on the source host it is not stopped | |
| 09:08:51 | sean-k-mooney | ack ya that cleaner | |
| 09:09:04 | sean-k-mooney | and the existing check for is it safe to evacute woudl also be checked | |
| 09:09:13 | gibi | yes | |
| 09:09:15 | sean-k-mooney | e.g. the heatbeat has been missed or you set force_down | |
| 09:09:54 | sean-k-mooney | i think johnthetubaguy expressed interest in this in the past | |
| 09:10:17 | sean-k-mooney | or at least supprot for the people that were askign for it in the past | |
| 09:11:09 | sean-k-mooney | oh that reminds me i guess we are not merging my default change? | |
| 09:11:34 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/830829 | |
| 09:12:36 | sean-k-mooney | i would either like to merge that before RC1 or after we create the stable branch | |
| 09:17:38 | sahid | thank you guys - are we agree to extend evacuate with target state (AsBefore, Stopped) ? Can I report a bug with destailled description or should I share a spec? | |
| 09:21:17 | sean-k-mooney | sahid: all api changes require a spec regardless of how trivial | |
| 09:21:25 | sean-k-mooney | this would need a spec and a new microverion | |
| 09:22:24 | sahid | ack I make this happen for A. | |
| 09:22:24 | sean-k-mooney | it can be a pretty short spec but there will at least be a conductor rpc change and likely a compute one too pass the target state | |
| 09:22:43 | sahid | sure no worries | |
| 09:23:52 | sean-k-mooney | sahid: the repo is open for sepc reviews so whenever you have time feel free to submit one | |
| 09:24:44 | sahid | +1 | |
| 09:29:37 | bauzas | sahid, gibi, sean-k-mooney: tbh, I think we discussed about the host-evacuate support before, and we said this was close to orchestration as a client could be doing it | |
| 09:29:58 | bauzas | so the consensus is to tell that some client or script could be doing it | |
| 09:30:10 | bauzas | (or Heat, or whatever else) | |
| 09:31:22 | sean-k-mooney | bauzas: yep | |
| 09:31:43 | sean-k-mooney | bauzas: but what we were discussiong regaring a spec wa allwoing the api to accept a target state to evacuate too | |
| 09:32:03 | sean-k-mooney | oneof:active,poweredoff or current | |
| 09:32:20 | sean-k-mooney | where current or not specified means what we do today | |
| 10:10:31 | zigo | gibi: oslo.concurrency 5.0.1 hangs when I try to run its unit tests, which is probably related to your patch (at: https://review.opendev.org/c/openstack/oslo.concurrency/+/855714 ). Any idea what's going on? Do I need the latest Eventlet (I know we're lagging one minor version behind)? | |
| 10:12:17 | zigo | Ah no, I'm even with 0.30.2, maybe that's why... | |
| 10:14:24 | gibi | zigo: do you know where it is hanging? which test acse? | |
| 10:14:25 | gibi | case | |
| 10:15:13 | zigo | Hard to tell... | |
| 10:16:05 | zigo | Last output was: https://paste.opendev.org/show/bsDsX7d0gw1SDAnLcRMA/ | |
| 10:16:20 | zigo | Then it hangged ... | |
| 10:16:36 | sean-k-mooney | whats that form? | |
| 10:16:49 | zigo | sean-k-mooney: building oslo.concurrency. | |
| 10:17:23 | sean-k-mooney | oh i disconnected and reconnected for a bit | |
| 10:17:32 | sean-k-mooney | missed the start of your conversation with gibi i think | |
| 10:18:11 | zigo | Problem is: I have autopkgtest issues in Eventlet ... 0.33.x :/ | |
| 10:18:47 | sean-k-mooney | https://github.com/openstack/oslo.concurrency/blob/5397838f4117300a509bff474dfcdd60b5993677/oslo_concurrency/tests/unit/test_processutils.py#L184-L201 | |
| 10:20:10 | sean-k-mooney | i see | |
| 10:20:28 | sean-k-mooney | so thats failing whiel building in some cases | |
| 10:20:58 | sean-k-mooney | im not really sure who/why | |
| 10:21:01 | gibi | zigo: could you point to how you run the unti tests? | |
| 10:21:31 | sean-k-mooney | the est is just concorting an instnace of an exeption class | |
| 10:21:58 | sean-k-mooney | asserting when you call str() on it that it contians the message | |
| 10:22:10 | gibi | there are unit tests in the repo that can be run with and without eventlet monkey patching but there are tests that can only run in eventlet | |
| 10:22:13 | gibi | hence the https://github.com/openstack/oslo.concurrency/blob/01cf2ffdf48c21f886b2aa3f766be5d268248c18/tox.ini#L14-L15 | |
| 10:22:30 | sean-k-mooney | maybe this print is the issue https://github.com/openstack/oslo.concurrency/blob/5397838f4117300a509bff474dfcdd60b5993677/oslo_concurrency/tests/unit/test_processutils.py#L203 | |
| 10:22:49 | zigo | I simply do this: | |
| 10:22:49 | zigo | PYTHON=python3 stestr run --subunit | subunit2pyunit | |
| 10:23:11 | gibi | I can imagine that if you run the eventlet aware test without eventlet monkey patching then the eventlet only test might missbehave | |
| 10:23:38 | zigo | I'll try further and let you know where it leads me. | |
| 10:24:07 | sean-k-mooney | if you do things like eventlet.spwawn directly without monkeypatching | |
| 10:24:17 | sean-k-mooney | you need to manually invoke the event loop to have it run | |
| 10:24:33 | sean-k-mooney | we saw that in the nova-api when we added scater gather | |
| 10:25:20 | gibi | zigo: based on that command line you run without eventlet monkey patching but you still run tests from https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/tests/unit/test_lockutils_eventlet.py | |
| 10:26:07 | gibi | zigo: you can try not running those test to see if that resolve the hang | |
| 10:26:30 | sean-k-mooney | you could fix them by adding eventlet.sleep(seconds=0) | |
| 10:26:48 | sean-k-mooney | i think that will make it work if not monkeypatched | |
| 10:27:05 | sean-k-mooney | but ya not runnign them would be better | |
| 10:27:24 | gibi | there is the place where the test monkey patches selectively https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/tests/__init__.py | |
| 10:28:10 | sean-k-mooney | https://github.com/openstack/oslo.concurrency/blob/5397838f4117300a509bff474dfcdd60b5993677/oslo_concurrency/tests/unit/test_lockutils_eventlet.py#L49 | |