| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-12 | |||
| 14:48:52 | dansmith | belmoreira: it's been like a couple years at this point.. :P | |
| 14:49:16 | dansmith | I dunno how reliable it is because I think it's very infrequently used, if that :/ | |
| 14:49:55 | sean-k-mooney | i think its in train | |
| 14:50:11 | sean-k-mooney | but ya i don tknow if any of our downstream custoemr use it | |
| 14:50:24 | sean-k-mooney | most of them are not multi cell | |
| 14:50:26 | belmoreira | honestly, I was searching for it just few minutes ago... in all releases after Rocky because I recall that mriedem was working on it | |
| 14:51:08 | belmoreira | but anyway... what I'm looking at is a way to live migrate not cold migrate | |
| 14:51:11 | sean-k-mooney | oh its ussuri https://specs.openstack.org/openstack/nova-specs/specs/ussuri/implemented/cross-cell-resize.html | |
| 14:51:27 | sean-k-mooney | belmoreira: right we basically agree to not implemnte live migration | |
| 14:51:34 | sean-k-mooney | effectivly ever | |
| 14:51:40 | dansmith | sean-k-mooney: did we? | |
| 14:51:44 | belmoreira | :) | |
| 14:51:59 | sean-k-mooney | we could but it would reuiqre assumeing conenctivty betwenn the hyperviors in diffenet cells | |
| 14:52:01 | dansmith | IIRC it was going to be more complicated, very little audience, and so had no plans | |
| 14:52:19 | sean-k-mooney | dansmith: well i thinke the connectivty was the main blocker | |
| 14:52:21 | dansmith | sure, but live migration already assumes connectivity between hosts that not everyone allows, even in one cel | |
| 14:52:36 | sean-k-mooney | i gues that is true | |
| 14:52:52 | belmoreira | my use case is a one off... I'm trying to get rid of nova-network | |
| 14:53:06 | dansmith | my recollection was "if there's large uptake of cells and this becomes a thing lots of people want, we can consider, but no plans until then" | |
| 14:53:08 | sean-k-mooney | belmoreira: if you have a need for it and or people to work on it i would be happy to review a spec | |
| 14:53:16 | dansmith | and so far, no such demand that I know of | |
| 14:53:29 | sean-k-mooney | dansmith: yep i think that is a fiar summary | |
| 14:53:43 | belmoreira | the hypothetical plan would be to live migrate the instances from a nova-network cell to a neutron cell | |
| 14:54:06 | dansmith | belmoreira: well, unlikely that would work anyway I think, without very special planning on our part, | |
| 14:54:15 | dansmith | which we obviously can't and won't do without the code being in master :) | |
| 14:55:08 | belmoreira | yeah... honestly I'm not asking for it... but it would be great if we can discuss the blockers... in case I need to deal with them | |
| 14:55:57 | belmoreira | the main problem in my view would be to make both source and dest talk (different conductors) and populate the DB | |
| 14:56:19 | dansmith | belmoreira: it would need to work the same way the cross-cell cold migration works, | |
| 14:56:28 | dansmith | which is the superconductor does that bridging from the nova perspective | |
| 14:56:52 | dansmith | i.e. it's the thing that creates the instance in the target cell and deletes it from the old cell when the time comes | |
| 14:57:27 | sean-k-mooney | dansmith: we might also need to proxy the conntion between the hypervirous via the the super condctor host or some other host if we coudl not asusme direct conenctivity | |
| 14:57:46 | belmoreira | in my case I can have direct connectivity | |
| 14:57:48 | sean-k-mooney | there are also complcaiton when volumes or ceph gets invovled | |
| 14:58:05 | sean-k-mooney | belmoreira: is your cinder/ceph storage shared accross cells | |
| 14:58:19 | belmoreira | yes, it is | |
| 14:58:46 | dansmith | sean-k-mooney: well, or we have to redesign the live migration flow to go through (super)conductor in a more push-pull sort of way | |
| 15:00:05 | sean-k-mooney | well i ment makign it so qemu on one host could talke to qemu on another via proxign the coneection. | |
| 15:01:23 | dansmith | oh, I dunno, I think you'd just make it a requirement that they can talk direct, | |
| 15:01:29 | dansmith | even if for a short time | |
| 15:01:55 | dansmith | implementing a well-performing proxy at the superconductor (which is likely just a container on a host) seems like more work than necessary | |
| 15:02:24 | sean-k-mooney | ya it does | |
| 15:02:35 | belmoreira | those are not limitations for me... | |
| 15:02:49 | sean-k-mooney | belmoreira: yep the were just limiation in general | |
| 15:02:59 | sean-k-mooney | e.g. if you map cells to edge sites | |
| 15:03:12 | sean-k-mooney | cross cell migration get a lot harder if its over the wan | |
| 15:04:38 | belmoreira | it looks like it's a lot of work... because its a live migration the instance entries need to be created in the dest cell DB, domain XML needs to be created, take care of the network logic, and possible ceph volumes attached... qemu starts the migration and it needs to be removed from the source DB, placement updated | |
| 15:04:55 | sean-k-mooney | i dont link config driven api behavior but i kind of feel like we woudl need a way to enabel/disabel this perhapes via the sschuler config in soem way | |
| 15:05:33 | sean-k-mooney | belmoreira: we would have to also update the instnace cell mapping in the api db | |
| 15:05:45 | belmoreira | true | |
| 15:05:50 | sean-k-mooney | which is parly why this need to be driven by the super conductor | |
| 15:05:57 | dansmith | sean-k-mooney: we disabled it via policy now | |
| 15:06:09 | sean-k-mooney | dansmith: cross cell live migration | |
| 15:06:29 | sean-k-mooney | how does the policy rule know its corsscell | |
| 15:07:06 | sean-k-mooney | are we actully doign a lookup of the host if we force the destionaion? | |
| 15:07:39 | belmoreira | Cross cell resize was added in Ussuri (i was searching for migration, that's why I missed it) | |
| 15:08:18 | dansmith | sean-k-mooney: we have a flag to the api that says "can cross cell boundaries" I think, and only allow that if policy passes | |
| 15:09:00 | sean-k-mooney | belmoreira: ah ya same code path internelly | |
| 15:12:13 | belmoreira | great... looks like I will be the oldest nova-network user around :) | |
| 15:12:38 | belmoreira | dansmith sean-k-mooney thanks a lot for the comments and insights | |
| 15:15:01 | dansmith | sean-k-mooney: well, I thought we expressed intent to the api that we wanted to cross, but apparently it's only the config flag, but yeah we check the policy deeper in compute/api to decide if the user is allowed | |
| 15:18:19 | sean-k-mooney | ya that would work. i dont think we modify the request body to request cross cell either, so ya it woudl have to be checked later in the compute api | |
| 15:35:18 | giblet | sean-k-mooney: Am I correct that nova uses the physnet of the neutron network in two cases 1) matching it with the physical_network property of the PciDevice (coming from the whitelist) during scheduling if there is InstancePciRequest 2) if numa aware vswitches are configured then matching against novas configuration | |
| 15:35:23 | giblet | https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/numa-aware-vswitches.html#id33 ? | |
| 15:36:26 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 15:48:33 | elodilles | bauwser: sorry, I forgot to ping you earlier about the ussuri release patch (as today is the Ussuri Transition Day o:)) if you could approve this that would be great :) https://review.opendev.org/c/openstack/releases/+/817226 | |
| 15:50:12 | bauwser | elodilles: done :) | |
| 15:52:26 | elodilles | bauwser: cool, thanks! \o/ I'll update the nova ussuri-em patch after this has merged | |
| 15:52:38 | bauwser | ;) | |
| 16:50:06 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 16:52:49 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 16:55:02 | elodilles | bauwser if you are still here: ussuri relese patch has merged and the ussuri-em patch is updated: https://review.opendev.org/c/openstack/releases/+/817606 | |
| 16:55:21 | bauwser | elodilles: last call, indeed. | |
| 16:56:51 | elodilles | bauwser: uh, didn't want to leave it to end of the day, but somehow forgot to ping you earlier :S sorry :/ | |
| 16:57:05 | bauwser | np :) | |
| 17:08:15 | artom | dansmith, hey, if we do an instance.save() for example, but no fields have changed, we don't actually do the DB update, correct? | |
| 17:16:04 | bauwser | elodilles: I could be wrong but I said 'no' to https://review.opendev.org/c/openstack/releases/+/817606 | |
| 17:16:57 | dansmith | artom: https://github.com/openstack/nova/blob/171138146a648d22474b7021ac730e26f03455f8/nova/objects/instance.py#L810-L811 | |
| 17:17:38 | artom | dansmith, oh, it's only in instance, not provided by base ovo? | |
| 17:17:53 | dansmith | artom: not all objects even have a save | |
| 17:18:03 | artom | Doh, right | |
| 17:18:14 | dansmith | artom: base ovo provides dirty tracking for the fields, but what you do with it is up to you | |
| 17:20:07 | artom | dansmith, yeah, makes sense | |
| 17:58:38 | sean-k-mooney | giblet: yes those are the two cases it uses it | |
| 18:01:02 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 18:12:44 | elodilles | bauwser: sorry, yes, so those patches are not relevant from release perspective :) | |
| 18:14:06 | elodilles | bauwser: why would we force customers to upgrade those packages on production systems because of some TOX and zuul and gerrit related changes? o:) | |
| 18:15:06 | sean-k-mooney[m] | which packages | |
| 18:15:23 | elodilles | osc-placement and python-novaclient | |
| 18:15:48 | elodilles | sean-k-mooney[m]: see this patch: https://review.opendev.org/c/openstack/releases/+/817606 | |
| 18:18:49 | sean-k-mooney[m] | the oscp placment tox changes wont alter what is installed | |
| 18:19:16 | sean-k-mooney[m] | it just uses the ussuri upper-constratings instead of master | |
| 18:22:26 | elodilles | yepp, exactly :) | |
| 18:23:08 | sean-k-mooney[m] | elodilles i would personally update the shas to the latest if we are doing a release anyway but i dont see any issues that would need a release to be packaged | |
| 18:23:48 | sean-k-mooney[m] | there is a fix for one func test in novaclient | |
| 18:23:56 | sean-k-mooney[m] | but again wont affect end users | |
| 18:24:46 | elodilles | sean-k-mooney[m]: the *-em tag is not a "release" it is a tag against the latest release. and releases should be meaningful for end users as far as i know | |
| 20:25:35 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 20:26:39 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 21:53:52 | opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test token expiration during live migration https://review.opendev.org/c/openstack/nova/+/817778 | |
| 23:00:24 | opendevreview | David Hill proposed openstack/nova stable/ussuri: Ensure MAC addresses characters are in the same case https://review.opendev.org/c/openstack/nova/+/817689 | |