| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-31 | |||
| 12:26:02 | brinzhang_ | yeah, I will try | |
| 12:27:31 | brinzhang_ | gibi: it's true, thanks for you explain ^^ | |
| 12:29:21 | openstackgerrit | Maciej Józefczyk proposed openstack/nova master: [WIP] Respect multiple segments in network https://review.opendev.org/716275 | |
| 12:30:36 | openstackgerrit | Maciej Józefczyk proposed openstack/nova master: [WIP] Respect multiple segments in network https://review.opendev.org/716275 | |
| 12:30:49 | gibi | bauzas: I only found two mentions in the nova doc about FFU | |
| 12:31:24 | bauzas | gibi: I'm writing a ML thread FWIW | |
| 12:31:58 | gibi | doc/source/user/cells.rst: be a gap for other deployment tools. Consider also the FFU case | |
| 12:31:58 | gibi | $ egrep -e "fast forward|FFU" -R doc/ | |
| 12:32:01 | gibi | doc/source/reference/upgrade-checks.rst: fast forward upgrading from Ocata to Rocky, something could have been | |
| 12:32:04 | gibi | bauzas: yeah thanks. | |
| 12:33:33 | gibi | bauzas: one way to remove reshape is to add an upgrade check | |
| 12:35:39 | bauzas | gibi: http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013709.html | |
| 12:37:10 | johnthetubaguy | are we not meant to be able to FFU between *any* release, like gibi said, we are probably just missing an upgrade check | |
| 12:37:58 | bauzas | gibi: mmm, that'd make sense | |
| 12:38:01 | openstackgerrit | Maciej Józefczyk proposed openstack/nova master: [WIP] Respect multiple segments in network https://review.opendev.org/716275 | |
| 12:38:18 | johnthetubaguy | bauzas: can your code be activated without any nova services running? | |
| 12:38:45 | bauzas | gibi: but since reshapes are made automatically when you restart nova-compute in Stein or later, that doesn't need an operator modification | |
| 12:39:03 | bauzas | johnthetubaguy: nope, you need to restart nova-compute, that's the main concern | |
| 12:39:18 | johnthetubaguy | bauzas: in which case we can't remove that code until we have one | |
| 12:39:29 | johnthetubaguy | when we then enforce to be run via an upgrade check | |
| 12:39:36 | gibi | bauzas: we need to make sure that all the compute was started up one during FFU _before_ Ussuri | |
| 12:39:37 | bauzas | and I think we started discussing on this kind of problem, but we haven't had an agreement *yet* | |
| 12:39:37 | johnthetubaguy | then following release we can drop the code | |
| 12:40:16 | gibi | yeah, I don't find any agreement | |
| 12:40:30 | gibi | but the above upgrade check based solution could be a good proposal | |
| 12:40:32 | bauzas | johnthetubaguy: the problem I have with this direction is that usually we write an upgrade check for asking operators to do *something* before upgrading | |
| 12:40:51 | bauzas | johnthetubaguy: in this specific case, what would it be ? restart n-cpu? | |
| 12:40:57 | johnthetubaguy | bauzas: we need to give the operators a way to do something though, we *cannot* requite nova-compute to be started | |
| 12:41:07 | johnthetubaguy | s/requite/require/ | |
| 12:41:20 | johnthetubaguy | at least that was my understanding of the situation | |
| 12:41:41 | bauzas | johnthetubaguy: well, I think we drafted this discussion somewhere sometimes in a PTG and we said it should be a separate module that would be run off nova services | |
| 12:42:04 | bauzas | it, being the reshape codes (with a trailing s, please note) | |
| 12:42:19 | bauzas | but we honestly never designed it | |
| 12:42:26 | sean-k-mooney | johnthetubaguy: requireing n-cpu to be started breaks ffu | |
| 12:42:30 | johnthetubaguy | I was fairly sure it had to be nova-manage, possibly running on each compute node... but yeah I guess we didn't | |
| 12:42:41 | johnthetubaguy | sean-k-mooney: exactly what I am saying, +1 | |
| 12:42:44 | sean-k-mooney | but only if we remove the code before x releases | |
| 12:42:50 | bauzas | and i don't disagree with it | |
| 12:43:13 | sean-k-mooney | e.g. we have to keep the reshape and code for ~3 release if we require you to start the agent | |
| 12:43:15 | bauzas | but given we never thought more about that, we never agreed on *when* and *how* to remove such reshape codes | |
| 12:43:21 | johnthetubaguy | I guess we can't remove the code till we design a way to get rid of it | |
| 12:43:52 | bauzas | agreed | |
| 12:44:03 | johnthetubaguy | cool | |
| 12:44:11 | sean-k-mooney | i think osa and ooo are the only tools that do ffu | |
| 12:44:34 | sean-k-mooney | and i think both only support 3 releases max so at most we should need to keep it for 3 releases | |
| 12:44:40 | johnthetubaguy | I kinda got the impression the packages folks were going to adopt it at some point | |
| 12:44:55 | sean-k-mooney | ffu | |
| 12:44:57 | bauzas | sean-k-mooney: the problem is that we need to leave this code for 5 releases then | |
| 12:45:03 | johnthetubaguy | yeah | |
| 12:45:22 | sean-k-mooney | honestly i thnk ffu was a mistake to begin with and we should have followed kolla anisbles lead and gone with rolling upgrades | |
| 12:45:28 | bauzas | ie. we can assume that people could FFU from Train to Victoria or W | |
| 12:45:47 | bauzas | actually, it's a bad example, nevermind | |
| 12:45:49 | sean-k-mooney | for me ffu was a fairly major design mistake | |
| 12:46:12 | bauzas | the worst case scenario is in my case : upgrade from Rocky to Ussuri | |
| 12:46:20 | johnthetubaguy | I think for FFU we basically have to say *any* to *any*, else its pointless | |
| 12:46:39 | johnthetubaguy | well, that or we only care about osa/ooo | |
| 12:46:40 | bauzas | so, at least in my case, Ussuri *has to* support FFU | |
| 12:47:06 | johnthetubaguy | bauzas: are you saying punt the conversation to the PTG? | |
| 12:47:09 | bauzas | I mean, only if we agree on a 3-release time window as a maximum | |
| 12:47:22 | bauzas | johnthetubaguy: I'm saying I'm shooting myself in the foot, yes | |
| 12:47:34 | johnthetubaguy | heh, I was just checking ;) | |
| 12:47:53 | sean-k-mooney | johnthetubaguy: redhat is moving ore ffu cycle to 2 release by the way | |
| 12:48:12 | bauzas | johnthetubaguy: I have concerns about how the virtual PTG would be productive, but let's be it | |
| 12:48:45 | johnthetubaguy | it depends how much work folks do ahead of time, it *can* work well, seen that in kolla | |
| 12:48:58 | sean-k-mooney | so every second release will be an LTS and the plan is to ffu between those, although that plan could change. the first case of this will be train to victorica | |
| 12:49:02 | ebbex | I'm having issues with live-migration on stable/stein, at first neutron was complaining about port_id existing on the target (even though status was inactive), so I removed the port_id in neutron.ml2_port_binding{s,_levels}. Now when trying to migrate I got this stack-trace; http://paste.openstack.org/show/791395/ and a vm_state error. Which databases and tables should I look at to fix this? | |
| 12:49:24 | bauzas | sean-k-mooney: the whole point of FFU is that we don't have *LTS* releases | |
| 12:49:37 | sean-k-mooney | bauzas: FFU encurages them | |
| 12:49:37 | bauzas | every release can be seen as a LTS one | |
| 12:49:40 | bauzas | no | |
| 12:49:43 | sean-k-mooney | yes | |
| 12:49:54 | sean-k-mooney | that is why we have so many people on osp 10 and 13 | |
| 12:50:09 | sean-k-mooney | it makes upgrading harder at least with ooo | |
| 12:50:09 | bauzas | every single user of FFU can just promote any release as an LTS | |
| 12:50:34 | bauzas | sure, for redhat, but what about other FFU users ? | |
| 12:50:47 | sean-k-mooney | its mainly just redhat | |
| 12:50:52 | sean-k-mooney | osa support it | |
| 12:50:53 | bauzas | we provided the tools for this, we can't just say 'heh, it's just redhat' | |
| 12:50:58 | bauzas | hah, jinxed | |
| 12:51:02 | sean-k-mooney | but they can do better upgrades without it | |
| 12:51:04 | johnthetubaguy | I like to think windows 10 adopted the same approach as kolla-ansible (although clearly factually incorrect) | |
| 12:51:27 | bauzas | I just think we entered a rabbit hole | |
| 12:52:03 | bauzas | honestly, we have to be pragmatic and solve the problem that johnthetubaguy stated : n-cpu restart shouldn't be a prereq | |
| 12:52:10 | bauzas | that's it | |
| 12:52:21 | sean-k-mooney | if we dont cap the limit on ffu we can never remove reshape code which i dont think is a resonable constratint | |
| 12:52:27 | bauzas | then we can argue on the tooling we would make | |
| 12:52:33 | johnthetubaguy | the question to ooo and osa, is runing a nova-manage command on every compute node a big deal or not | |
| 12:52:51 | bauzas | sean-k-mooney: if reshape code isn't tied to a specific nova module, I'm fine with it | |
| 12:53:10 | johnthetubaguy | bauzas: does reshape need details from the compute node? | |
| 12:53:15 | johnthetubaguy | I was assuming it does | |
| 12:53:16 | gibi | bauzas: is it expensive to keep the reshape code? | |
| 12:53:21 | sean-k-mooney | johnthetubaguy: in general yes | |
| 12:53:24 | bauzas | johnthetubaguy: you know what ? when we started to draft it, I was even on a stance about a specific other library | |
| 12:53:39 | sean-k-mooney | somethimes that info will be stored in the db but not always | |
| 12:53:44 | johnthetubaguy | could we make operators supply enough enough to not need that info? | |
| 12:53:59 | bauzas | johnthetubaguy: again, the problem is that we haven't designed the expected API between the compute service and the reshape code | |
| 12:54:22 | johnthetubaguy | +1 gibi's question | |
| 12:54:22 | sean-k-mooney | in the numa case im expecting most of the inf would be in the resouce track numa toplogy blob so it might not need to have acess to the compute node but that depends on if we remove the info form the db or not | |
| 12:54:42 | bauzas | I personnally wrote my own reshape code by writing some hard dependencies | |
| 12:54:53 | bauzas | (IIRC, it's even requiring libvirtd to run) | |