| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-12-12 | |||
| 16:30:28 | bauzas | fwiw, I'm okay with verifying some link every week during our meeting about how many job runs failed | |
| 16:30:31 | sean-k-mooney | you said local images dont work | |
| 16:30:39 | sean-k-mooney | did you make sure to use vmdks | |
| 16:30:42 | sean-k-mooney | instead of qcow | |
| 16:30:46 | bauzas | so even if we don't make them voting, we could continue to check those jobs continue to work | |
| 16:31:08 | fwiesel | sean-k-mooney: Sure, we only run with vmdks. | |
| 16:31:25 | sean-k-mooney | ok i was wondering if it was a simple format issue | |
| 16:31:35 | sean-k-mooney | feel free to file a bug with details when you have time | |
| 16:31:49 | bauzas | fwiesel: do you know we changed the VDMK types ? | |
| 16:32:08 | sean-k-mooney | o we blocked one of the times right | |
| 16:32:15 | bauzas | https://bugs.launchpad.net/nova/+bug/1996188 for the context | |
| 16:32:24 | fwiesel | bauzas: No, I don't. Thanks for the info | |
| 16:32:34 | bauzas | so you now need to pass an allowed list of vdmk types | |
| 16:32:50 | fwiesel | Ah, no. That one is fine... The same check is in cinder, and it works with boot from volume | |
| 16:32:55 | bauzas | maybe this is the root cause, maybe not | |
| 16:33:16 | bauzas | fwiesel: nova has its own config option | |
| 16:33:29 | bauzas | https://review.opendev.org/c/openstack/nova/+/871612 | |
| 16:33:41 | fwiesel | Sure, but we use the image subformat that isn't blocked | |
| 16:33:49 | bauzas | cool | |
| 16:33:57 | fwiesel | (streamOptimized or something) | |
| 16:34:01 | bauzas | was more a fyi, just in case | |
| 16:34:16 | bauzas | the vdmk bug hit me before :) | |
| 16:34:45 | bauzas | and when we had it, noone was around for telling us whether it was a problem for vmwareapi :) | |
| 16:34:57 | bauzas | anyway | |
| 16:35:01 | bauzas | shouldn't be a problem | |
| 16:35:19 | bauzas | fwiesel: thanks for the report, greatly appreciated | |
| 16:35:30 | fwiesel | you're welcome | |
| 16:35:36 | bauzas | fwiesel: someone also freaked out in the mailing list | |
| 16:35:45 | bauzas | I haven't replied but you could | |
| 16:35:57 | fwiesel | bauzas: Good idea, I will | |
| 16:36:11 | fwiesel | bauzas: openstack-discuss? | |
| 16:36:18 | bauzas | oh sean-k-mooney did | |
| 16:36:35 | bauzas | fwiesel: yup https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/DLTQ2KHQPFD4S7LLYKTWUPNFXRSTRTHU/ | |
| 16:36:54 | sean-k-mooney | ya i just said what we dicussed before | |
| 16:37:07 | sean-k-mooney | i.e. we wont remvoe anything until at least m2 depending on the ci status | |
| 16:37:17 | sean-k-mooney | and advies that while peopel can deprecate support | |
| 16:37:18 | bauzas | fwiw, I see a good progress | |
| 16:37:20 | sean-k-mooney | they shoudl not remove it | |
| 16:37:28 | bauzas | but let's continue to discuss it every week | |
| 16:37:33 | sean-k-mooney | yep | |
| 16:37:44 | bauzas | anyway, good talks | |
| 16:38:54 | bauzas | fwiw, I also advertised in the previous OpenInfraLive episode about what we agreed at the PTG and I explained verbally our current state, which is good for telling 'if you care about something, come tell us' | |
| 16:39:22 | bauzas | taking vmwareapi non-removal as an example of a working effort | |
| 16:39:46 | bauzas | anyway, time flis | |
| 16:39:50 | bauzas | flies* | |
| 16:39:53 | bauzas | #topic Open discussion | |
| 16:39:58 | bauzas | we have two topics | |
| 16:40:07 | bauzas | that we punted from last week | |
| 16:40:12 | bauzas | (artom) Specless blueprint for persistent mdevs | |
| 16:40:17 | bauzas | https://blueprints.launchpad.net/nova/+spec/persistent-mdevs | |
| 16:40:24 | bauzas | artom: around ? | |
| 16:40:40 | artom | Heya | |
| 16:41:16 | artom | To basically yeah - from memory, this would be limited to the libvirt driver, the idea is to persist mdevs when an instance is booted that uses an mdev | |
| 16:41:24 | bauzas | I'm maybe opiniated, so I won't really tell a lot, but I think this is a simple specless feature that only touches how our virt driver is creating a mdev | |
| 16:41:32 | artom | So that in case of a host reboot, the instances can come back without operator intervention | |
| 16:42:02 | artom | There might be operator intervention necessary to manually clean mdevs in certain cases | |
| 16:42:04 | bauzas | I don't see any upgrade concerns about it, the fact is that we will start persisting mdevs upon reboot by every compute that's upgraded | |
| 16:42:15 | artom | Because the mdevs would outlive their instances and host reboots | |
| 16:42:37 | artom | So for instance, changing the enabled mdev types (after draining the host), the operator would need to clean up the old mdevs | |
| 16:42:52 | sean-k-mooney | one thing you need to be careful of | |
| 16:42:56 | bauzas | artom: surely this will require a releasenote and some upstream docs, but this doesn't require to add a new DB model or anything about RPC | |
| 16:42:57 | gibi | as draining the host will not remove mdevs? | |
| 16:43:08 | sean-k-mooney | is on restating nova to a version with this support | |
| 16:43:19 | sean-k-mooney | if we have vms using mdevs created via sysfs | |
| 16:43:49 | sean-k-mooney | we need to supprot cretign the libvirt nodedev object to persist them | |
| 16:44:05 | bauzas | sean-k-mooney: I see the upgrade path for persisting the mdevs to restart the instances | |
| 16:44:12 | sean-k-mooney | i.e. we need to supprot upgrade in place without restarts | |
| 16:44:21 | sean-k-mooney | why? | |
| 16:44:33 | sean-k-mooney | we shoudl not need vm downtime or move operations | |
| 16:44:41 | bauzas | oh my bad, I was wrong | |
| 16:44:50 | bauzas | the mdev would already be created | |
| 16:44:54 | sean-k-mooney | yep | |
| 16:44:57 | sean-k-mooney | and in use by the vm | |
| 16:45:09 | sean-k-mooney | we jsut need to create the mdev with the same uuid in the libvirt api | |
| 16:45:12 | sean-k-mooney | to have it persited | |
| 16:45:19 | bauzas | so, I think this feature requires some upgrade doc that explains how to persist the mdecv | |
| 16:45:33 | sean-k-mooney | well nova can do it | |
| 16:45:34 | bauzas | I mean, an admin doc | |
| 16:45:42 | sean-k-mooney | but we can have n upgade doc to cover how this works | |
| 16:46:10 | sean-k-mooney | i would hope its litrally just update the nova-compute binary adn restart the compute agent | |
| 16:46:15 | bauzas | sean-k-mooney: do you think of a nova-compute startup method that would check every single mdev and would persist it ? | |
| 16:46:15 | sean-k-mooney | no other upgrade impact | |
| 16:46:29 | sean-k-mooney | init host can reconsile what we expect based on teh current xmls | |
| 16:46:41 | sean-k-mooney | ya | |
| 16:46:45 | sean-k-mooney | that is what i was thinking | |
| 16:46:48 | bauzas | that's an additional effort, sure, still specless I think | |
| 16:46:57 | sean-k-mooney | we can defer that to impelmation review if we like | |
| 16:47:13 | sean-k-mooney | but i woudl ike to see upgrade in place support with or without a hard reboot i guess | |
| 16:47:15 | bauzas | sounds acceptable o me | |
| 16:47:37 | bauzas | we already have a broken method that's run on init_host | |
| 16:47:48 | bauzas | we could amend it to persist the mdev instead | |
| 16:48:12 | bauzas | (ie. delete and recreate it using libvirt API with the same uuid) | |
| 16:48:38 | bauzas | artom: does that sound reasonable to you ? | |
| 16:49:06 | artom | Sorry, reading back, in multiple places at the same time | |
| 16:49:28 | bauzas | IRC meetings are good, but people do many things at the same time :) | |
| 16:49:39 | bauzas | I wish we could be more in sync somehow :) | |
| 16:50:34 | bauzas | given we only have 10 mins left and anther procedural approval to do | |
| 16:50:41 | bauzas | lemme summarize | |
| 16:51:06 | bauzas | 1/ the feature will ensure that every new mdev to create will use the libvirt API | |
| 16:51:38 | bauzas | 2/ at compute restart, the implementation will check every mdev that's created and will recreate it using the libvirt API | |