| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-04-23 | |||
| 14:49:38 | sean-k-mooney | so it will have one version that is never updated for all fo rhel/centos 8/x | |
| 14:49:47 | sean-k-mooney | * 8.X | |
| 14:50:45 | kashyap | sean-k-mooney: Why do you "minimum version"? E.g. https://review.opendev.org/#/c/716526/3/deployment/nova/nova-compute-container-puppet.yaml | |
| 14:51:22 | kashyap | sean-k-mooney: They update it to whatever all the Compute nodes were tested with | |
| 14:51:44 | sean-k-mooney | kashyap: when train released on 8.0 ooo still need to be able to install that on 8.1 or 8.2 later | |
| 14:52:04 | kashyap | sean-k-mooney: Sure, it will work; why would that be any problem? | |
| 14:52:15 | elod | lyarwood: looks good to me. +W'd | |
| 14:52:25 | sean-k-mooney | but if you used the max version for that majory version there would be issues with live migraion later if you scaled out a cludete ther started on 8.0 with new node on 8.1 | |
| 14:52:26 | kashyap | If you set the machine type to 8.0; you can of course install it on 8.1 and 8.2 | |
| 14:52:46 | lyarwood | elod: thanks | |
| 14:53:08 | kashyap | sean-k-mooney: Yes, the version is chosen in a way that the operator must ensure the Compute nodes are capable of it | |
| 14:53:40 | kashyap | sean-k-mooney: Recall, the version in the machien type matches CentOS|RHEL minor version - it is pretty clear to figure out the problem due to host OS version mismatch... | |
| 14:53:46 | sean-k-mooney | yep wich i ugess is the max verion supported by the oldes hypervior | |
| 14:54:02 | sean-k-mooney | which would be the verion form 8.0 in this case | |
| 14:54:41 | kashyap | Right. | |
| 14:54:46 | kashyap | We clear? :) | |
| 14:57:40 | kashyap | I take it we are. | |
| 15:02:51 | sean-k-mooney | kashyap: yep sorry was talking dowstream | |
| 15:03:09 | sean-k-mooney | and ya we are on the same page regarding the requirements here | |
| 15:03:31 | kashyap | No problem; thanks | |
| 15:10:21 | lyarwood | kashyap / sean-k-mooney ; silly question but why wouldn't we use the q35 alias all the time downstream? | |
| 15:10:42 | kashyap | lyarwood: Live migration compatibility | |
| 15:10:51 | lyarwood | kashyap: but LM wouldn't switch between the two surely? | |
| 15:10:53 | kashyap | lyarwood: I have an example written down, with a scenario, so you can see it plainly | |
| 15:11:04 | kashyap | Let me post it somewhere, so you can see it, and possibly punch holes in it | |
| 15:12:06 | lyarwood | ah wait so QEMU would just blindly use the alias on the dest as well? | |
| 15:12:13 | lyarwood | even if it pointed to a newer version | |
| 15:12:43 | lyarwood | sorry I assumed libvirt/QEMU would translate the alias into an actual versioned q35 machine type and stick with that for the life cycle of the domain | |
| 15:14:58 | kashyap | lyarwood: Yes | |
| 15:15:32 | kashyap | lyarwood: If you try to migrate a guest with RHEL-7.7 machine type (running on RHEL-7 COmpute node) to a RHEL-7.6 Compute node -- it fails spectacularly | |
| 15:15:45 | kashyap | Because 7.6 host obviously doesn't know about 7.7 :) | |
| 15:16:47 | lyarwood | right but my point is more that forward migration from 7.6 to 7.7 just using the q35 alias could also fail | |
| 15:17:03 | lyarwood | as the domain would jump from the 7.6 to 7.7 version of the machine type | |
| 15:17:05 | kashyap | lyarwood: https://kashyapc.fedorapeople.org/versioned-machine-types-and-live-migration-gotcha.txt | |
| 15:17:08 | lyarwood | well maybe not fail but things could change | |
| 15:17:27 | kashyap | lyarwood: Forward migration fomr 7.6 to 7.7 will _not_ fail | |
| 15:17:28 | kashyap | Why? | |
| 15:17:55 | kashyap | Because, 7.7 host will have all the older (7.6, 7.5, 7.4) machine types precisely for live migration compat | |
| 15:18:07 | kashyap | lyarwood: I had the above text somewhere in the spec, but deleted it, for "brevity" | |
| 15:18:23 | kashyap | artom: Since you also spent non-trivial time reviewing the spec, you might benefit from reading this: https://kashyapc.fedorapeople.org/versioned-machine-types-and-live-migration-gotcha.txt | |
| 15:18:42 | lyarwood | It will now be _impossible_ to migrate a guest from a RHEL 7.6-based | |
| 15:18:43 | lyarwood | compute node to a RHEL 7.7-based — because RHEL 7.6 won't know about | |
| 15:18:45 | lyarwood | the RHEL 7.7's machine type. | |
| 15:18:56 | lyarwood | kashyap: ^ that was my point sorry | |
| 15:19:25 | lyarwood | and the workaround appears to be to pin to the oldest version | |
| 15:19:33 | kashyap | Yep | |
| 15:19:40 | lyarwood | but I just don't get how this works downstream across multple releases | |
| 15:19:49 | lyarwood | we'd basically need to pin to the first version used in the env | |
| 15:20:55 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Fix list rendering in the accelerator support doc https://review.opendev.org/721846 | |
| 15:21:58 | lyarwood | surely we should work out and persist the versioned machine type during a migration? | |
| 15:22:24 | lyarwood | so if the source is 8.1 and using the q35 alias ensure it uses that on the dest | |
| 15:22:36 | lyarwood | but all new instances use the 8.2 q35 type? | |
| 15:22:56 | kashyap | lyarwood: For downstream, yes: you need to pick a versioned machine type and set it across the nodes before deplyoying. | |
| 15:22:59 | aarents | lyarwood: kashyap we hit this kind of issues.. but Im not the guy who work on it I will check how we dealt with that.. | |
| 15:23:01 | lyarwood | kashyap: sorry I'm talking out loud here, could you add the LM section back into the spec and I'll add words there. | |
| 15:23:26 | kashyap | lyarwood: Sure, let me add that bit in the spec in a separate section; it's important | |
| 15:23:30 | lyarwood | kashyap: yeah the issue with that is you can never move from it | |
| 15:24:16 | kashyap | lyarwood: Hang on - you _can_ move; it's a step-by-step process: | |
| 15:25:35 | kashyap | lyarwood: Err, my "Here's the problem" has a bug! | |
| 15:25:41 | kashyap | It should read the _other_ way round. | |
| 15:25:43 | kashyap | Let me update it | |
| 15:25:59 | lyarwood | kashyap: kk, lets get the spec updated and chat there :) | |
| 15:26:02 | kashyap | You cannot migrte from 7.7 to 7.6 | |
| 15:26:14 | kashyap | But you can jolly well migrate from 7.6 to 7.7 :) | |
| 15:26:15 | kashyap | Yes | |
| 15:26:17 | kashyap | A sec | |
| 15:30:33 | kashyap | aarents: Noted :) | |
| 15:30:33 | artom | I feel like this whole mess could be solved if we tracked machine type on a per-instance basis | |
| 15:31:00 | artom | Though the problem that raises is live-migration from an N+1 to an N compute | |
| 15:31:14 | artom | The dest N might now know the machine type, as it's newer | |
| 15:31:37 | kashyap | Per-instance tracking is just too damn fragile and messy | |
| 15:31:47 | kashyap | I'd say per-Compute based is better for debugging, too. | |
| 15:31:51 | artom | Maybe, but how/why? | |
| 15:31:55 | kashyap | We should not forget the Operator's predicmanet | |
| 15:32:02 | kashyap | s/predicmanet/predicament/ | |
| 15:32:10 | artom | This whole discussion shows that per-compute isn't super clean either :) | |
| 15:32:22 | kashyap | artom: The live migration from N+1 to N always remains a problem. | |
| 15:32:54 | artom | True | |
| 15:33:06 | artom | Could we not incorporate machine type into our LM checks? | |
| 15:33:10 | kashyap | Can you tell more what do we gain more per-instance tracking solve? Just asking sincerely... :) | |
| 15:42:44 | sean-k-mooney | kashyap: if we dont track the machine type and you upgreade the host the vm might not work after a hardreboot | |
| 15:43:14 | sean-k-mooney | e.g. if it had a conflciting image peroperty like hw_vif_model=e1000 or hw_disk_bus=ide | |
| 15:44:09 | sean-k-mooney | so if its not set in the image and is just being chosen by our default or the host config then we need to recored it to not break existing instnace on upgrade | |
| 15:44:35 | sean-k-mooney | hopefuly that makes sense | |
| 15:50:55 | gibi | nova meeting starts in 10 minutes on #openstack-nova | |
| 15:51:12 | gibi | nova meeting starts in 10 minutes on #openstack-meeting-3 | |
| 15:51:18 | gibi | /o\ | |
| 15:52:30 | lyarwood | elod / melwitt: https://review.opendev.org/#/c/720587/ - I think this is ready right? We aren't waiting on actual failures to show up first in stable/stein? | |
| 15:53:11 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: Make RBD imagebackend flatten method idempotent https://review.opendev.org/708615 | |
| 15:53:20 | kashyap | sean-k-mooney: Will respond in a few | |
| 15:53:57 | sean-k-mooney | kashyap: no rush | |
| 15:53:59 | kashyap | sean-k-mooney: lyarwood: So, I just double-confirmed with upstream QEMU: as I guessed, depsite KM-long threads, there's no agreement whether to not add new 'pc' variants or not -- it is strictly a RHEL-only decsion | |
| 15:54:31 | kashyap | So, I'll add "suitable words" | |
| 15:54:31 | kashyap | And also other long-term distributions (SUSE, et al) | |
| 15:54:33 | kashyap | (In the end this spec might even not be needed ... but at least it allows to flesh out ideas) | |
| 15:55:18 | sean-k-mooney | kashyap: yep. ill try to review it more properly on monday but at least it s place to collect and discuss the alternitives | |
| 15:56:46 | kashyap | sean-k-mooney: Sure, yeah. It at least keeps the ideas in one place | |
| 15:58:00 | openstackgerrit | Ghanshyam Mann proposed openstack/python-novaclient master: [Community goal] Update contributor documentation https://review.opendev.org/722391 | |
| 15:58:07 | openstackgerrit | Ghanshyam Mann proposed openstack/os-vif master: [Community goal] Update contributor documentation https://review.opendev.org/722392 | |
| 15:58:14 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: [Community goal] Update contributor documentation https://review.opendev.org/722393 | |