| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-31 | |||
| 12:52:03 | bauzas | honestly, we have to be pragmatic and solve the problem that johnthetubaguy stated : n-cpu restart shouldn't be a prereq | |
| 12:52:10 | bauzas | that's it | |
| 12:52:21 | sean-k-mooney | if we dont cap the limit on ffu we can never remove reshape code which i dont think is a resonable constratint | |
| 12:52:27 | bauzas | then we can argue on the tooling we would make | |
| 12:52:33 | johnthetubaguy | the question to ooo and osa, is runing a nova-manage command on every compute node a big deal or not | |
| 12:52:51 | bauzas | sean-k-mooney: if reshape code isn't tied to a specific nova module, I'm fine with it | |
| 12:53:10 | johnthetubaguy | bauzas: does reshape need details from the compute node? | |
| 12:53:15 | johnthetubaguy | I was assuming it does | |
| 12:53:16 | gibi | bauzas: is it expensive to keep the reshape code? | |
| 12:53:21 | sean-k-mooney | johnthetubaguy: in general yes | |
| 12:53:24 | bauzas | johnthetubaguy: you know what ? when we started to draft it, I was even on a stance about a specific other library | |
| 12:53:39 | sean-k-mooney | somethimes that info will be stored in the db but not always | |
| 12:53:44 | johnthetubaguy | could we make operators supply enough enough to not need that info? | |
| 12:53:59 | bauzas | johnthetubaguy: again, the problem is that we haven't designed the expected API between the compute service and the reshape code | |
| 12:54:22 | johnthetubaguy | +1 gibi's question | |
| 12:54:22 | sean-k-mooney | in the numa case im expecting most of the inf would be in the resouce track numa toplogy blob so it might not need to have acess to the compute node but that depends on if we remove the info form the db or not | |
| 12:54:42 | bauzas | I personnally wrote my own reshape code by writing some hard dependencies | |
| 12:54:53 | bauzas | (IIRC, it's even requiring libvirtd to run) | |
| 12:55:03 | johnthetubaguy | sounds like we could write something that we backport to train that fixes this on the api node | |
| 12:55:21 | sean-k-mooney | johnthetubaguy: for the vgpus no | |
| 12:55:35 | johnthetubaguy | well, then we need that info from the operator | |
| 12:55:38 | sean-k-mooney | johnthetubaguy: we dont track those in the db | |
| 12:55:49 | bauzas | I need to look at the reshape spaghetti code for the vgpus | |
| 12:55:55 | bauzas | and see what assumptions I made | |
| 12:56:34 | johnthetubaguy | and we would need something from placement, as its API will not be up? | |
| 12:56:37 | bauzas | johnthetubaguy: we *at least* need placement-api to be up and running | |
| 12:56:42 | johnthetubaguy | not allowed | |
| 12:56:47 | bauzas | yeah I know | |
| 12:56:51 | johnthetubaguy | would need placement-manage to take something | |
| 12:56:52 | bauzas | but that's what we merged | |
| 12:57:03 | johnthetubaguy | yep yep | |
| 12:57:12 | johnthetubaguy | we merged something that violates the rules for FFU | |
| 12:57:20 | bauzas | correct | |
| 12:57:46 | bauzas | well, technically, you need to start n-cpu and placement in Stein | |
| 12:57:53 | bauzas | when you FFU from Rocky to Train | |
| 12:57:56 | bauzas | that's the prereq | |
| 12:58:30 | bauzas | (but honestly, since Ussuri works, you can FFU to Ussuri and do the reshape then) | |
| 12:58:48 | johnthetubaguy | sounds like we keep it till at least Z | |
| 12:58:53 | bauzas | oh yeagh | |
| 12:59:00 | bauzas | or we write something to fix that in V | |
| 12:59:05 | sean-k-mooney | i know that ffu in k8s land or rather openshift has stopper versions | |
| 12:59:06 | bauzas | or W | |
| 12:59:08 | johnthetubaguy | sounds like we have better things to do | |
| 12:59:35 | sean-k-mooney | e.g. you can fast forwared but they publish a list of verions you must stop at and start the kublet before continuing | |
| 12:59:44 | johnthetubaguy | sean-k-mooney: they also break their public API most releases | |
| 12:59:46 | sean-k-mooney | that is basically what we have | |
| 12:59:49 | bauzas | the problem is that this little assumption that we could be in non-nested world makes my life on feature development much harder | |
| 13:00:06 | sean-k-mooney | johnthetubaguy: well its technically versioned so you can use the older version but yes | |
| 13:00:06 | bauzas | which is something I can expose to my management | |
| 13:00:14 | bauzas | and say that we need to work on a solution in V or W | |
| 13:00:18 | johnthetubaguy | sean-k-mooney: till they remove it, yep | |
| 13:00:41 | johnthetubaguy | not saying its bad, just different | |
| 13:00:47 | johnthetubaguy | they have a different context | |
| 13:01:42 | johnthetubaguy | sounds like if we wait a few more releases, it will sort its-self out one way or the other | |
| 13:01:44 | sean-k-mooney | yes and no, they have a different approch. we have similar but not idential scopes. they do applciation lifecyle management better and openstack is better sutied to infrastuture | |
| 13:01:48 | johnthetubaguy | like gibi said, maybe the cost isn't that high | |
| 13:02:05 | johnthetubaguy | sean-k-mooney: agreed | |
| 13:02:45 | johnthetubaguy | personally, running something on every compute node sounds a lot like starting up n-cpu | |
| 13:02:58 | johnthetubaguy | so really we just shouldn't do that kind of thing | |
| 13:03:37 | sean-k-mooney | well if your calling if FFU | |
| 13:03:48 | sean-k-mooney | i like how kolla-ansible does there upgrades | |
| 13:03:52 | johnthetubaguy | so we need some comprimise where the easy thing (just reshape after upgrade) works for most folks, and the last few folks have some workaround (supply info about the shape of their hosts to some nova-manage) | |
| 13:03:59 | johnthetubaguy | but that's just my take | |
| 13:04:07 | bauzas | gibi: I know you're on a meeting, but do we already have an etherpad for the virtual Victoria PTG ? | |
| 13:04:20 | gibi | bauzas: yes, https://etherpad.openstack.org/p/nova-victoria-ptg | |
| 13:04:24 | bauzas | thanks | |
| 13:04:53 | sean-k-mooney | gibi: one thing i think we need to plan for are some cross project sessions too | |
| 13:05:27 | gibi | sean-k-mooney: true. Do you have topic for cross-project discussion? | |
| 13:05:51 | sean-k-mooney | not specifcally but i was concerned that that topic had not come up | |
| 13:05:58 | gibi | sean-k-mooney: I think you can simply add that to the current etherpad (just start a new section that indicate that this is a cross project topic that needs anohter team) | |
| 13:06:21 | gibi | you mean people are not thinking about cross project topics because there is no place to put them? | |
| 13:06:22 | sean-k-mooney | ther are things like the unified limits work johnthetubaguy et al are working on that i think needs to continue but i did not really have anything i wanted to drive | |
| 13:07:05 | sean-k-mooney | gibi: i think when we talk about a virtual ptg many teams are just thinking about there team, not the collabative cross team stuff we also do at the ptg | |
| 13:07:11 | gibi | sean-k-mooney: OK, I hope the affected people will propose topics | |
| 13:07:53 | bauzas | that reminds me I have to provide my toughts on two ML threads from gibi | |
| 13:10:15 | gibi | sean-k-mooney: added a Cross project topics section | |
| 13:11:20 | sean-k-mooney | gibi: one topic that comes to mind actully is the schduling supprot for routed networks | |
| 13:11:30 | sean-k-mooney | or network aware schduling in general | |
| 13:11:51 | sean-k-mooney | that said im not sure i plan to work on that in the near term so im not going to add it | |
| 13:12:07 | sean-k-mooney | but i know the neuton team are interested in that topic | |
| 13:13:12 | gibi | sean-k-mooney: ack, that is actually one thing I can add as I started looking at that from nova perspective | |
| 13:15:14 | gibi | sean-k-mooney: added one item for routed nets | |
| 13:16:15 | sean-k-mooney | gibi: cool, i kind of wrote up how i think it should work in the comment on matts poc | |
| 13:16:36 | sean-k-mooney | but that was more involed then the minium viable change | |
| 13:17:46 | gibi | sean-k-mooney: if you have input then do not hesitate to write it up, I kept that patch on the minimum viable change level but we can agree to do a bigger change in V | |
| 13:18:02 | gibi | sean-k-mooney: my idea was that the current minimum viable patch does not need neutron change | |
| 13:18:05 | sean-k-mooney | unless you are doing sriov nova just assumes that the destination host will be able to support the networking that is requested. this has always been the case and has worked because for the most part ops make there vxlan and vlans span the datacenter but it has never really been a correct assumtion to make. | |
| 13:19:42 | johnthetubaguy | routed networks make the opposite assumption, you usually don't get to pick the IP until you know where has compute resources (unless you are re-using a port, when there are other issues in play). | |
| 13:20:11 | sean-k-mooney | johnthetubaguy: correct and nova currently is not aware of those constraints | |
| 13:20:34 | sean-k-mooney | which is why live migration basially does not work unless you spcify a host in the same segment | |
| 13:20:50 | johnthetubaguy | yeah, only does a safety check | |
| 13:21:17 | sean-k-mooney | my toughts on the topic are partly summerised in https://review.opendev.org/#/c/656885/ in my top comment on 25 of feb | |
| 13:21:20 | johnthetubaguy | live-migration scheduling is an interesting one... I don't really see a use case for it, but I hey. | |
| 13:22:00 | sean-k-mooney | johnthetubaguy: you dont see a usecase for allowing live migration to move the vm to another host in the same network segment where it can keep its ip | |
| 13:23:10 | johnthetubaguy | operationally, for our customers, its super rare you want it "to go somewhere the scheduler picks", sure we can make up cases where its needed | |
| 13:23:45 | sean-k-mooney | well that is the oppisite feedback we are geting from the neutron folks | |
| 13:24:07 | johnthetubaguy | I think there is a bigger issue there, you want to define live-migratable pools, etc, etc. | |
| 13:24:18 | johnthetubaguy | well I look forward to better understanding those folks use cases | |
| 13:25:06 | sean-k-mooney | johnthetubaguy: yes i tired to adress the more generic case to include vlan awarenes and ip pools modeled in placment in my comment | |
| 13:26:39 | sean-k-mooney | basically using placment aggreates to model pool affinity with sharing resouce providers modelign ip capsity,vlan/vxlan ids(as segmenation_ids) ectra and traits modeling the type of network | |