| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-12-12 | |||
| 16:31:49 | bauzas | fwiesel: do you know we changed the VDMK types ? | |
| 16:32:08 | sean-k-mooney | o we blocked one of the times right | |
| 16:32:15 | bauzas | https://bugs.launchpad.net/nova/+bug/1996188 for the context | |
| 16:32:24 | fwiesel | bauzas: No, I don't. Thanks for the info | |
| 16:32:34 | bauzas | so you now need to pass an allowed list of vdmk types | |
| 16:32:50 | fwiesel | Ah, no. That one is fine... The same check is in cinder, and it works with boot from volume | |
| 16:32:55 | bauzas | maybe this is the root cause, maybe not | |
| 16:33:16 | bauzas | fwiesel: nova has its own config option | |
| 16:33:29 | bauzas | https://review.opendev.org/c/openstack/nova/+/871612 | |
| 16:33:41 | fwiesel | Sure, but we use the image subformat that isn't blocked | |
| 16:33:49 | bauzas | cool | |
| 16:33:57 | fwiesel | (streamOptimized or something) | |
| 16:34:01 | bauzas | was more a fyi, just in case | |
| 16:34:16 | bauzas | the vdmk bug hit me before :) | |
| 16:34:45 | bauzas | and when we had it, noone was around for telling us whether it was a problem for vmwareapi :) | |
| 16:34:57 | bauzas | anyway | |
| 16:35:01 | bauzas | shouldn't be a problem | |
| 16:35:19 | bauzas | fwiesel: thanks for the report, greatly appreciated | |
| 16:35:30 | fwiesel | you're welcome | |
| 16:35:36 | bauzas | fwiesel: someone also freaked out in the mailing list | |
| 16:35:45 | bauzas | I haven't replied but you could | |
| 16:35:57 | fwiesel | bauzas: Good idea, I will | |
| 16:36:11 | fwiesel | bauzas: openstack-discuss? | |
| 16:36:18 | bauzas | oh sean-k-mooney did | |
| 16:36:35 | bauzas | fwiesel: yup https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/DLTQ2KHQPFD4S7LLYKTWUPNFXRSTRTHU/ | |
| 16:36:54 | sean-k-mooney | ya i just said what we dicussed before | |
| 16:37:07 | sean-k-mooney | i.e. we wont remvoe anything until at least m2 depending on the ci status | |
| 16:37:17 | sean-k-mooney | and advies that while peopel can deprecate support | |
| 16:37:18 | bauzas | fwiw, I see a good progress | |
| 16:37:20 | sean-k-mooney | they shoudl not remove it | |
| 16:37:28 | bauzas | but let's continue to discuss it every week | |
| 16:37:33 | sean-k-mooney | yep | |
| 16:37:44 | bauzas | anyway, good talks | |
| 16:38:54 | bauzas | fwiw, I also advertised in the previous OpenInfraLive episode about what we agreed at the PTG and I explained verbally our current state, which is good for telling 'if you care about something, come tell us' | |
| 16:39:22 | bauzas | taking vmwareapi non-removal as an example of a working effort | |
| 16:39:46 | bauzas | anyway, time flis | |
| 16:39:50 | bauzas | flies* | |
| 16:39:53 | bauzas | #topic Open discussion | |
| 16:39:58 | bauzas | we have two topics | |
| 16:40:07 | bauzas | that we punted from last week | |
| 16:40:12 | bauzas | (artom) Specless blueprint for persistent mdevs | |
| 16:40:17 | bauzas | https://blueprints.launchpad.net/nova/+spec/persistent-mdevs | |
| 16:40:24 | bauzas | artom: around ? | |
| 16:40:40 | artom | Heya | |
| 16:41:16 | artom | To basically yeah - from memory, this would be limited to the libvirt driver, the idea is to persist mdevs when an instance is booted that uses an mdev | |
| 16:41:24 | bauzas | I'm maybe opiniated, so I won't really tell a lot, but I think this is a simple specless feature that only touches how our virt driver is creating a mdev | |
| 16:41:32 | artom | So that in case of a host reboot, the instances can come back without operator intervention | |
| 16:42:02 | artom | There might be operator intervention necessary to manually clean mdevs in certain cases | |
| 16:42:04 | bauzas | I don't see any upgrade concerns about it, the fact is that we will start persisting mdevs upon reboot by every compute that's upgraded | |
| 16:42:15 | artom | Because the mdevs would outlive their instances and host reboots | |
| 16:42:37 | artom | So for instance, changing the enabled mdev types (after draining the host), the operator would need to clean up the old mdevs | |
| 16:42:52 | sean-k-mooney | one thing you need to be careful of | |
| 16:42:56 | bauzas | artom: surely this will require a releasenote and some upstream docs, but this doesn't require to add a new DB model or anything about RPC | |
| 16:42:57 | gibi | as draining the host will not remove mdevs? | |
| 16:43:08 | sean-k-mooney | is on restating nova to a version with this support | |
| 16:43:19 | sean-k-mooney | if we have vms using mdevs created via sysfs | |
| 16:43:49 | sean-k-mooney | we need to supprot cretign the libvirt nodedev object to persist them | |
| 16:44:05 | bauzas | sean-k-mooney: I see the upgrade path for persisting the mdevs to restart the instances | |
| 16:44:12 | sean-k-mooney | i.e. we need to supprot upgrade in place without restarts | |
| 16:44:21 | sean-k-mooney | why? | |
| 16:44:33 | sean-k-mooney | we shoudl not need vm downtime or move operations | |
| 16:44:41 | bauzas | oh my bad, I was wrong | |
| 16:44:50 | bauzas | the mdev would already be created | |
| 16:44:54 | sean-k-mooney | yep | |
| 16:44:57 | sean-k-mooney | and in use by the vm | |
| 16:45:09 | sean-k-mooney | we jsut need to create the mdev with the same uuid in the libvirt api | |
| 16:45:12 | sean-k-mooney | to have it persited | |
| 16:45:19 | bauzas | so, I think this feature requires some upgrade doc that explains how to persist the mdecv | |
| 16:45:33 | sean-k-mooney | well nova can do it | |
| 16:45:34 | bauzas | I mean, an admin doc | |
| 16:45:42 | sean-k-mooney | but we can have n upgade doc to cover how this works | |
| 16:46:10 | sean-k-mooney | i would hope its litrally just update the nova-compute binary adn restart the compute agent | |
| 16:46:15 | bauzas | sean-k-mooney: do you think of a nova-compute startup method that would check every single mdev and would persist it ? | |
| 16:46:15 | sean-k-mooney | no other upgrade impact | |
| 16:46:29 | sean-k-mooney | init host can reconsile what we expect based on teh current xmls | |
| 16:46:41 | sean-k-mooney | ya | |
| 16:46:45 | sean-k-mooney | that is what i was thinking | |
| 16:46:48 | bauzas | that's an additional effort, sure, still specless I think | |
| 16:46:57 | sean-k-mooney | we can defer that to impelmation review if we like | |
| 16:47:13 | sean-k-mooney | but i woudl ike to see upgrade in place support with or without a hard reboot i guess | |
| 16:47:15 | bauzas | sounds acceptable o me | |
| 16:47:37 | bauzas | we already have a broken method that's run on init_host | |
| 16:47:48 | bauzas | we could amend it to persist the mdev instead | |
| 16:48:12 | bauzas | (ie. delete and recreate it using libvirt API with the same uuid) | |
| 16:48:38 | bauzas | artom: does that sound reasonable to you ? | |
| 16:49:06 | artom | Sorry, reading back, in multiple places at the same time | |
| 16:49:28 | bauzas | IRC meetings are good, but people do many things at the same time :) | |
| 16:49:39 | bauzas | I wish we could be more in sync somehow :) | |
| 16:50:34 | bauzas | given we only have 10 mins left and anther procedural approval to do | |
| 16:50:41 | bauzas | lemme summarize | |
| 16:51:06 | bauzas | 1/ the feature will ensure that every new mdev to create will use the libvirt API | |
| 16:51:38 | bauzas | 2/ at compute restart, the implementation will check every mdev that's created and will recreate it using the libvirt API | |
| 16:52:11 | artom | OK, yeah, I think that makes sense, though the mechanics of persisting existing transient mdevs are less obvious to me at this time | |
| 16:52:25 | bauzas | 3/ documentation will address the fact that operator needs some cleanup (unpersisting the mdevs) in case they want to change the type, additional to the fact they need to drain vgpu instances from that host | |
| 16:52:38 | sean-k-mooney | i would hope its just generateing an xml and asking livbirt to create it | |
| 16:52:54 | bauzas | artom: don't be afraid, I exactly see what, where and how to do it | |
| 16:52:57 | sean-k-mooney | it should see that it already exist and i hope just write the mdevctl file | |
| 16:53:07 | sean-k-mooney | we can do this at the end of the seriese | |
| 16:53:12 | sean-k-mooney | to not blcoke the over all feature | |
| 16:53:32 | bauzas | based on those 3 bullet points, I don't see anything that requires a spec | |