| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-01 | |||
| 08:53:32 | sean-k-mooney | local deleted yes | |
| 08:54:15 | sean-k-mooney | i tought we did for evacuated instance but stephenfin did not find one when he quickly search on the triage call | |
| 08:54:26 | sean-k-mooney | it might exist already but proably not | |
| 08:54:37 | gibi | lyarwood: yepp we have _cleanup_running_deleted_instances | |
| 08:54:58 | gibi | sean-k-mooney: there is no periodic for evac cleanup I've checked | |
| 08:56:03 | gibi | we have _error_out_instances_whose_build_was_interrupted during init_host as well | |
| 08:56:22 | gibi | that also someting that is only done in init_host | |
| 08:56:29 | gibi | as might need to be done after forece up | |
| 08:57:27 | sean-k-mooney | on a related note there was also https://review.opendev.org/c/openstack/nova/+/627765 before | |
| 08:58:00 | sean-k-mooney | in this case looking at orpahned instance which are deleted instance that have been purged from the database before that periodic runs | |
| 08:58:30 | gibi | anyhow we are divergin. Lee only wants a backportable thing now | |
| 08:58:43 | lyarwood | Lee always wants a backportable thing ;) | |
| 08:59:09 | sean-k-mooney | yes to block more latent vms in thsi state | |
| 08:59:32 | lyarwood | but yeah I think this API change is backportable, a periodic could also be but adding a RPC call from the API during force up wouldn't be | |
| 08:59:53 | sean-k-mooney | lyarwood: i assume we woudl return a 409 | |
| 08:59:58 | lyarwood | yup | |
| 09:00:04 | lyarwood | that's how I've written it at the moment | |
| 09:00:10 | sean-k-mooney | proably with that is its a new return code | |
| 09:00:16 | gibi | I still don't like trying to build in safety into a workflow that we marked unsafe at the beginning. But if you frame the whole thing in a way that before nova assumes responsibility again about this compute node after a force_down we do some consistency check, then I can be convinced to remove my -1 | |
| 09:00:22 | sean-k-mooney | which is normally a microverison bump | |
| 09:00:29 | lyarwood | yeah but AFAIK we don't require microversion bumps for new error codes | |
| 09:00:39 | sean-k-mooney | i think we do | |
| 09:00:40 | lyarwood | I've asked gmann about this before | |
| 09:00:56 | sean-k-mooney | it think that only if we are converting from a 500 | |
| 09:01:09 | sean-k-mooney | to the flow chart | |
| 09:02:10 | lyarwood | sean-k-mooney: I was just looking for that, can you share the link? | |
| 09:02:18 | sean-k-mooney | https://docs.openstack.org/nova/latest/contributor/microversions.html | |
| 09:02:23 | lyarwood | ta | |
| 09:02:33 | sean-k-mooney | so yes we need a microversion if you use 409 | |
| 09:02:37 | sean-k-mooney | so 400? | |
| 09:02:38 | gibi | I think we need to keep it 400 for a backportable fix | |
| 09:02:50 | gibi | current clients are not prepared for 409 | |
| 09:02:58 | gibi | so it is breaking them | |
| 09:03:08 | sean-k-mooney | yep | |
| 09:04:03 | lyarwood | kk understood, with a TODO to move to 409 with a new microversion in Xena? | |
| 09:05:01 | sean-k-mooney | am sure we could | |
| 09:05:18 | gibi | hm | |
| 09:05:19 | sean-k-mooney | as long as we have a good error message | |
| 09:05:24 | gibi | An obvious regression bug in an admin-only API where the bug can still be fixed upstream on active stable branches. Admin-only APIs are less of a concern for interoperability and generally a regression in behavior can be dealt with as a bug fix when the documentation clearly shows the API behavior was unexpectedly regressed. See 3 for an example. Intentional behavior changes to an admin-only | |
| 09:05:24 | sean-k-mooney | i dont know if its needed | |
| 09:05:24 | gibi | we have have this as well | |
| 09:05:30 | gibi | API do require a microversion, like the 2.53 microversion for example. | |
| 09:05:59 | gibi | nvm, this bug is not a regression | |
| 09:06:12 | gibi | it is a new behavior | |
| 09:06:17 | sean-k-mooney | correct and the other exemption https://docs.openstack.org/nova/latest/contributor/microversions.html#id3 | |
| 09:06:26 | sean-k-mooney | also does not apply to 409 | |
| 09:06:43 | sean-k-mooney | also this is not a 500 | |
| 09:07:04 | gibi | lyarwood: yeah, let's keep a todo for Xena | |
| 09:07:50 | sean-k-mooney | lyarwood: you coudl just write both patches now. technially master is now xena | |
| 09:08:15 | sean-k-mooney | although we might want to hold off api microverion bumps until the release is actully done | |
| 09:08:35 | lyarwood | yup I'll get it posted later today before the break (./me is off until Tuesday after today). | |
| 09:08:37 | gibi | yepp, master is not fully open to Xena yet | |
| 09:08:48 | lyarwood | thanks both :) | |
| 09:09:19 | sean-k-mooney | ya ill be on pto till tuesday too | |
| 09:11:48 | gibi | I guess it is true for most of us | |
| 09:12:22 | sean-k-mooney | lyarwood: by the way partly while im off and partly early next week i plann to dismantel my home cloud and reinstall it and do some hardware tweeks | |
| 09:12:41 | sean-k-mooney | do you need any data form your vms | |
| 09:12:50 | lyarwood | sean-k-mooney: ack, nope I don't nuke away | |
| 09:13:29 | sean-k-mooney | cool ill be backing up a few thign but one of the change ill be doing is swaping my current cinder lvm sotrage for ceph so ill be easing most of the stroage | |
| 09:14:15 | sean-k-mooney | also doing an os reinstall moving form cenots 8 to stream wroked for a while but now im getting some repo conflictis so doing a reinstall to fix that | |
| 09:14:41 | lyarwood | huh I had assumed this was all running on Ubuntu tbh | |
| 09:14:51 | lyarwood | but cool | |
| 09:14:57 | sean-k-mooney | hehe well it will be soon | |
| 09:15:17 | sean-k-mooney | stephenfin: convicned me to try centos 8 for it then it lifecycle changed | |
| 09:15:42 | stephenfin | don't dare try to shift the blame to me - this is all on you :P | |
| 09:15:53 | sean-k-mooney | it is | |
| 09:16:00 | stephenfin | your fault for listening to me ;) | |
| 09:16:04 | sean-k-mooney | i also listend to you about tryign fedora on my laptop | |
| 09:16:13 | sean-k-mooney | i should have know better | |
| 09:16:20 | stephenfin | ha! | |
| 09:16:21 | stephenfin | fair | |
| 09:16:35 | sean-k-mooney | but ya am the version of container d that is ship in stream is not happy with docker | |
| 09:16:36 | kashyap | It's not all that bad ;-) | |
| 09:16:51 | sean-k-mooney | so i had to pin the packages | |
| 09:17:03 | stephenfin | mainly for multiple Python versions without external repos | |
| 09:17:07 | kashyap | Also, a gentle reminder: many virt and kernel bugs get first fixed in Fedora | |
| 09:17:09 | sean-k-mooney | i have hit a few other things like that that im hoping to avoid | |
| 09:17:28 | sean-k-mooney | kashyap: yep but i very realy hit those | |
| 09:17:33 | stephenfin | kashyap: You'd swear you worked for Red Hat or something 0:) | |
| 09:17:43 | kashyap | stephenfin: LOL, it's not about Red Hat, really :) | |
| 09:17:56 | sean-k-mooney | kashyap: what i have hit is selinux being unhappy with me on fedora alot | |
| 09:17:57 | kashyap | stephenfin: I'm speaking with my upstream hat, really :) | |
| 09:18:25 | lyarwood | sean-k-mooney: there's a command and a shiny website for that problem | |
| 09:18:40 | lyarwood | makes someone cry however so you might not want to try it | |
| 09:18:51 | sean-k-mooney | set it to permissive mode | |
| 09:19:01 | kashyap | stephenfin: Also, I closely w/ the virt upstreams and downstreams to the point that I feel part of those teams too :) | |
| 09:19:24 | lyarwood | sean-k-mooney: https://stopdisablingselinux.com/ | |
| 09:19:30 | kashyap | sean-k-mooney: It's not about bugs, per se. Even new virt features first land in Fedora, BTW :) | |
| 09:19:36 | kashyap | And kernel, of course | |
| 09:19:51 | lyarwood | but yeah selinux can be a royal PITA | |
| 09:20:15 | sean-k-mooney | lyarwood: well i have tried using the command to updat the policy that you get prometed with when there is a failure by the way | |
| 09:20:56 | kashyap | lyarwood: Go tell that to Red Hat It's part of the value prop ;-) | |
| 09:21:17 | sean-k-mooney | kashyap: sure but i dont want bleeding edge for my infra i was new but stable which is why i normally go with debiab/ubuntu distros with the mainline kernel | |
| 09:21:33 | lyarwood | it can add value and still be a PITA to use ;) | |
| 09:21:43 | sean-k-mooney | centos with some tweeks i coudl live with | |
| 09:21:54 | sean-k-mooney | fedora not so much for anything i want to not have to maintain too much | |
| 09:22:01 | kashyap | lyarwood: Heh, sure | |
| 09:22:48 | kashyap | lyarwood: It took me and RHT virt team several weeks to debug a crazy VMs+container interaction bug. | |
| 09:23:02 | sean-k-mooney | kashyap: lyarwood by the way the main thing that anowyed me about selinux recently is it does not allow you to use iso form your home directoy to create vms in virt-manager | |
| 09:23:06 | kashyap | lyarwood: Here's my summary notes, for your "bedtime SELinux reading": https://kashyapc.fedorapeople.org/SELinux_libvirt_and_QEMU_in_a_container.html | |