| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-31 | |||
| 12:42:04 | bauzas | it, being the reshape codes (with a trailing s, please note) | |
| 12:42:19 | bauzas | but we honestly never designed it | |
| 12:42:26 | sean-k-mooney | johnthetubaguy: requireing n-cpu to be started breaks ffu | |
| 12:42:30 | johnthetubaguy | I was fairly sure it had to be nova-manage, possibly running on each compute node... but yeah I guess we didn't | |
| 12:42:41 | johnthetubaguy | sean-k-mooney: exactly what I am saying, +1 | |
| 12:42:44 | sean-k-mooney | but only if we remove the code before x releases | |
| 12:42:50 | bauzas | and i don't disagree with it | |
| 12:43:13 | sean-k-mooney | e.g. we have to keep the reshape and code for ~3 release if we require you to start the agent | |
| 12:43:15 | bauzas | but given we never thought more about that, we never agreed on *when* and *how* to remove such reshape codes | |
| 12:43:21 | johnthetubaguy | I guess we can't remove the code till we design a way to get rid of it | |
| 12:43:52 | bauzas | agreed | |
| 12:44:03 | johnthetubaguy | cool | |
| 12:44:11 | sean-k-mooney | i think osa and ooo are the only tools that do ffu | |
| 12:44:34 | sean-k-mooney | and i think both only support 3 releases max so at most we should need to keep it for 3 releases | |
| 12:44:40 | johnthetubaguy | I kinda got the impression the packages folks were going to adopt it at some point | |
| 12:44:55 | sean-k-mooney | ffu | |
| 12:44:57 | bauzas | sean-k-mooney: the problem is that we need to leave this code for 5 releases then | |
| 12:45:03 | johnthetubaguy | yeah | |
| 12:45:22 | sean-k-mooney | honestly i thnk ffu was a mistake to begin with and we should have followed kolla anisbles lead and gone with rolling upgrades | |
| 12:45:28 | bauzas | ie. we can assume that people could FFU from Train to Victoria or W | |
| 12:45:47 | bauzas | actually, it's a bad example, nevermind | |
| 12:45:49 | sean-k-mooney | for me ffu was a fairly major design mistake | |
| 12:46:12 | bauzas | the worst case scenario is in my case : upgrade from Rocky to Ussuri | |
| 12:46:20 | johnthetubaguy | I think for FFU we basically have to say *any* to *any*, else its pointless | |
| 12:46:39 | johnthetubaguy | well, that or we only care about osa/ooo | |
| 12:46:40 | bauzas | so, at least in my case, Ussuri *has to* support FFU | |
| 12:47:06 | johnthetubaguy | bauzas: are you saying punt the conversation to the PTG? | |
| 12:47:09 | bauzas | I mean, only if we agree on a 3-release time window as a maximum | |
| 12:47:22 | bauzas | johnthetubaguy: I'm saying I'm shooting myself in the foot, yes | |
| 12:47:34 | johnthetubaguy | heh, I was just checking ;) | |
| 12:47:53 | sean-k-mooney | johnthetubaguy: redhat is moving ore ffu cycle to 2 release by the way | |
| 12:48:12 | bauzas | johnthetubaguy: I have concerns about how the virtual PTG would be productive, but let's be it | |
| 12:48:45 | johnthetubaguy | it depends how much work folks do ahead of time, it *can* work well, seen that in kolla | |
| 12:48:58 | sean-k-mooney | so every second release will be an LTS and the plan is to ffu between those, although that plan could change. the first case of this will be train to victorica | |
| 12:49:02 | ebbex | I'm having issues with live-migration on stable/stein, at first neutron was complaining about port_id existing on the target (even though status was inactive), so I removed the port_id in neutron.ml2_port_binding{s,_levels}. Now when trying to migrate I got this stack-trace; http://paste.openstack.org/show/791395/ and a vm_state error. Which databases and tables should I look at to fix this? | |
| 12:49:24 | bauzas | sean-k-mooney: the whole point of FFU is that we don't have *LTS* releases | |
| 12:49:37 | sean-k-mooney | bauzas: FFU encurages them | |
| 12:49:37 | bauzas | every release can be seen as a LTS one | |
| 12:49:40 | bauzas | no | |
| 12:49:43 | sean-k-mooney | yes | |
| 12:49:54 | sean-k-mooney | that is why we have so many people on osp 10 and 13 | |
| 12:50:09 | sean-k-mooney | it makes upgrading harder at least with ooo | |
| 12:50:09 | bauzas | every single user of FFU can just promote any release as an LTS | |
| 12:50:34 | bauzas | sure, for redhat, but what about other FFU users ? | |
| 12:50:47 | sean-k-mooney | its mainly just redhat | |
| 12:50:52 | sean-k-mooney | osa support it | |
| 12:50:53 | bauzas | we provided the tools for this, we can't just say 'heh, it's just redhat' | |
| 12:50:58 | bauzas | hah, jinxed | |
| 12:51:02 | sean-k-mooney | but they can do better upgrades without it | |
| 12:51:04 | johnthetubaguy | I like to think windows 10 adopted the same approach as kolla-ansible (although clearly factually incorrect) | |
| 12:51:27 | bauzas | I just think we entered a rabbit hole | |
| 12:52:03 | bauzas | honestly, we have to be pragmatic and solve the problem that johnthetubaguy stated : n-cpu restart shouldn't be a prereq | |
| 12:52:10 | bauzas | that's it | |
| 12:52:21 | sean-k-mooney | if we dont cap the limit on ffu we can never remove reshape code which i dont think is a resonable constratint | |
| 12:52:27 | bauzas | then we can argue on the tooling we would make | |
| 12:52:33 | johnthetubaguy | the question to ooo and osa, is runing a nova-manage command on every compute node a big deal or not | |
| 12:52:51 | bauzas | sean-k-mooney: if reshape code isn't tied to a specific nova module, I'm fine with it | |
| 12:53:10 | johnthetubaguy | bauzas: does reshape need details from the compute node? | |
| 12:53:15 | johnthetubaguy | I was assuming it does | |
| 12:53:16 | gibi | bauzas: is it expensive to keep the reshape code? | |
| 12:53:21 | sean-k-mooney | johnthetubaguy: in general yes | |
| 12:53:24 | bauzas | johnthetubaguy: you know what ? when we started to draft it, I was even on a stance about a specific other library | |
| 12:53:39 | sean-k-mooney | somethimes that info will be stored in the db but not always | |
| 12:53:44 | johnthetubaguy | could we make operators supply enough enough to not need that info? | |
| 12:53:59 | bauzas | johnthetubaguy: again, the problem is that we haven't designed the expected API between the compute service and the reshape code | |
| 12:54:22 | johnthetubaguy | +1 gibi's question | |
| 12:54:22 | sean-k-mooney | in the numa case im expecting most of the inf would be in the resouce track numa toplogy blob so it might not need to have acess to the compute node but that depends on if we remove the info form the db or not | |
| 12:54:42 | bauzas | I personnally wrote my own reshape code by writing some hard dependencies | |
| 12:54:53 | bauzas | (IIRC, it's even requiring libvirtd to run) | |
| 12:55:03 | johnthetubaguy | sounds like we could write something that we backport to train that fixes this on the api node | |
| 12:55:21 | sean-k-mooney | johnthetubaguy: for the vgpus no | |
| 12:55:35 | johnthetubaguy | well, then we need that info from the operator | |
| 12:55:38 | sean-k-mooney | johnthetubaguy: we dont track those in the db | |
| 12:55:49 | bauzas | I need to look at the reshape spaghetti code for the vgpus | |
| 12:55:55 | bauzas | and see what assumptions I made | |
| 12:56:34 | johnthetubaguy | and we would need something from placement, as its API will not be up? | |
| 12:56:37 | bauzas | johnthetubaguy: we *at least* need placement-api to be up and running | |
| 12:56:42 | johnthetubaguy | not allowed | |
| 12:56:47 | bauzas | yeah I know | |
| 12:56:51 | johnthetubaguy | would need placement-manage to take something | |
| 12:56:52 | bauzas | but that's what we merged | |
| 12:57:03 | johnthetubaguy | yep yep | |
| 12:57:12 | johnthetubaguy | we merged something that violates the rules for FFU | |
| 12:57:20 | bauzas | correct | |
| 12:57:46 | bauzas | well, technically, you need to start n-cpu and placement in Stein | |
| 12:57:53 | bauzas | when you FFU from Rocky to Train | |
| 12:57:56 | bauzas | that's the prereq | |
| 12:58:30 | bauzas | (but honestly, since Ussuri works, you can FFU to Ussuri and do the reshape then) | |
| 12:58:48 | johnthetubaguy | sounds like we keep it till at least Z | |
| 12:58:53 | bauzas | oh yeagh | |
| 12:59:00 | bauzas | or we write something to fix that in V | |
| 12:59:05 | sean-k-mooney | i know that ffu in k8s land or rather openshift has stopper versions | |
| 12:59:06 | bauzas | or W | |
| 12:59:08 | johnthetubaguy | sounds like we have better things to do | |
| 12:59:35 | sean-k-mooney | e.g. you can fast forwared but they publish a list of verions you must stop at and start the kublet before continuing | |
| 12:59:44 | johnthetubaguy | sean-k-mooney: they also break their public API most releases | |
| 12:59:46 | sean-k-mooney | that is basically what we have | |
| 12:59:49 | bauzas | the problem is that this little assumption that we could be in non-nested world makes my life on feature development much harder | |
| 13:00:06 | sean-k-mooney | johnthetubaguy: well its technically versioned so you can use the older version but yes | |
| 13:00:06 | bauzas | which is something I can expose to my management | |