| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-09-11 | |||
| 12:31:39 | sean-k-mooney | the OOM reaper runs on a per numa basis | |
| 12:31:58 | sean-k-mooney | so even if you have lots of free swap you can still have OOM issues | |
| 12:32:12 | sean-k-mooney | or free ram on the wrong numa node | |
| 12:33:48 | lajoskatona | bauzas: https://review.opendev.org/c/openstack/nova/+/928903 , pushed an update for qemu/libvirt support matrix, as I see no change in it in C and D, just to be clear for end-users | |
| 12:34:08 | sahid | sean-k-mooney: sean-k-mooney: do you remeber the microversion to use, even when i try to live-migrate on the specified host, I receive the error nova valid host was found | |
| 12:35:37 | sean-k-mooney | force was added in 2.30 and removed in 2.68 so 2.30-2.67 | |
| 12:36:02 | sean-k-mooney | sahid: note that force skips schdluer filters | |
| 12:36:28 | sean-k-mooney | it just checks the host exists, placement should still confirm that there is enouch suport for basic resouces | |
| 12:38:25 | sean-k-mooney | note that the compute status filter has been mandaory for many releases | |
| 12:38:28 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L242-L255 | |
| 12:38:46 | sean-k-mooney | so where that is not configurabel there is no way to schdule to a disabeld host by design | |
| 12:40:11 | sean-k-mooney | ah i miss rememeberd we intoduced this as a mandatoy prefilter 5 years ago | |
| 12:40:13 | sean-k-mooney | https://github.com/openstack/nova/commit/168d34c8d1161dc4d62493e194819297e079bb51 | |
| 12:40:28 | sean-k-mooney | so since placment because mandatory there is not way by design to schduler to disabled computes | |
| 13:38:01 | bauzas | sean-k-mooney: thanks for your work on the "vfat is non-bootable so we can skip it" series, just +2d saying this is fine given oslo will support vfat checks shortly | |
| 13:38:23 | bauzas | if not, I'd be afraid of another attack vector | |
| 13:38:48 | sean-k-mooney | there is an oslo patch for it reference form the 3rd patch | |
| 13:38:57 | bauzas | yes I know, hence my +2 | |
| 13:39:22 | sean-k-mooney | im also considerign doing a quick poc of both makeing epmeral disk unformated and removing the backing files form them | |
| 13:39:53 | sean-k-mooney | with that said i dont really have much time to look at either but since im already looking at that code i was going to spend an hour or two and see if they would be easy to do for epoxy | |
| 13:40:49 | sean-k-mooney | the reason i choose bootable as the new flag is i think tis reasonabel to say in the qemu driver that non-bootable disk should not have backing files ever. | |
| 13:41:09 | sean-k-mooney | but i have not really tought about all the implications yet so i didnt want to include that in this inital work | |
| 13:43:06 | bauzas | no worries, given by 2025.1 we will be able to look at vfat, we could remove the skip for non-bootable disks | |
| 13:43:48 | sean-k-mooney | we could thats definetly a diesign disucsison we shoudl have early in 2025.1 | |
| 13:44:16 | sean-k-mooney | in my head im kind fo grouping this with the geneal "state of the image backends" topic | |
| 13:44:44 | sean-k-mooney | i think this is just one of the many workitems that go into improving that going forward | |
| 14:18:12 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: Respect supplied arguments in novncproxy_base_url https://review.opendev.org/c/openstack/nova/+/928839 | |
| 14:32:02 | opendevreview | Kaloyan Kotlarski proposed openstack/nova master: libvirt: Fetch the available vCPUs from the respective cgroup https://review.opendev.org/c/openstack/nova/+/927474 | |
| 15:02:13 | sahid | alian | |
| 15:02:16 | sahid | alianz | |
| 15:23:27 | dansmith | sean-k-mooney: safe sounds okay to me | |
| 15:24:24 | sean-k-mooney | ok i can go change that how would you feel about using that for skiping the backing file too? | |
| 15:24:30 | sean-k-mooney | in a seperate patch obviously | |
| 15:24:53 | sean-k-mooney | i.e. not haveing swap an ephmeral have backing files with qcow backend | |
| 15:25:25 | sean-k-mooney | dansmith: ill fix the typoes ectra but were there any other concerns | |
| 15:25:42 | sean-k-mooney | ill repin the patch after the our internal calls end | |
| 15:27:12 | dansmith | sean-k-mooney: I have concerns about how many places you had to make that change :P | |
| 15:27:18 | dansmith | but that's imagebackend's fault not yours | |
| 15:33:44 | sean-k-mooney | ya at least 1 of those i was really not expecting | |
| 15:33:52 | sean-k-mooney | we have a lot of duplication | |
| 15:55:36 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add support for showing finish_time https://review.opendev.org/c/openstack/nova/+/928933 | |
| 16:06:12 | opendevreview | Dan Smith proposed openstack/nova stable/2023.1: [stable-only]: Port qcow2v2 support from oslo https://review.opendev.org/c/openstack/nova/+/928937 | |
| 16:57:38 | bauzas | we need a second core for https://review.opendev.org/c/openstack/nova/+/928662 please :) | |
| 16:59:46 | sean-k-mooney | that looks correct to me | |
| 17:01:17 | bauzas | thanks | |
| 19:03:15 | opendevreview | sean mooney proposed openstack/nova master: [DNM] testing rdo experimental jobs https://review.opendev.org/c/openstack/nova/+/927762 | |
| 19:05:18 | opendevreview | sean mooney proposed openstack/nova master: Add functional repoducer for ephemeral disks https://review.opendev.org/c/openstack/nova/+/928310 | |
| 19:05:19 | opendevreview | sean mooney proposed openstack/nova master: only safety check bootable files created from glance https://review.opendev.org/c/openstack/nova/+/928829 | |
| 19:05:19 | opendevreview | sean mooney proposed openstack/nova master: adapt to vfat support in oslo.utils https://review.opendev.org/c/openstack/nova/+/928462 | |
| 21:12:57 | opendevreview | Matthew Heler proposed openstack/nova master: Fix regression with live migration on shared storage https://review.opendev.org/c/openstack/nova/+/928970 | |
| 21:18:15 | supamatt | Can I get reviews on this? https://review.opendev.org/c/openstack/nova/+/928970 It's a regression fix to how live migration should function on shared storage. | |
| #openstack-nova - 2024-09-12 | |||
| 01:20:53 | opendevreview | melanie witt proposed openstack/nova master: DNM try minimalistic imagebackend refactor https://review.opendev.org/c/openstack/nova/+/925635 | |
| 05:00:22 | opendevreview | melanie witt proposed openstack/nova master: DNM try minimalistic imagebackend refactor https://review.opendev.org/c/openstack/nova/+/925635 | |
| 06:42:40 | gibi | bauzas: supamatt's bug feels like an RC candidate https://review.opendev.org/c/openstack/nova/+/928970 | |
| 06:43:04 | gibi | bauzas: I've added it to the tracking etherpad and I will review it properly after a coffee | |
| 06:43:19 | gibi | supamatt: thanks for bringing this to our attention | |
| 07:15:40 | gibi | supamatt: I have doubts about the problem the code pointed out by the report and the fix does not effect the setting of the destroy_disks variable that is reponsible to signal deletion of files from the disk at cleanup | |
| 07:23:13 | gibi | it seems nobody is around so I will update the bug report with my doubt/questions | |
| 07:43:43 | gibi | done | |
| 07:47:21 | bauzas | gibi: ack, sorry, I had trouble with my home network | |
| 07:48:21 | gibi | bauzas: no worries | |
| 07:48:49 | bauzas | gibi: this is on 2024.1 | |
| 07:48:55 | bauzas | Caracal so | |
| 07:49:04 | gibi | bauzas: it was a backport from master | |
| 07:49:07 | gibi | merged this cycle | |
| 07:49:12 | bauzas | technically, we could deliver RC1 and then backport that one in a .z release | |
| 07:49:28 | gibi | https://review.opendev.org/c/openstack/nova/+/909806 | |
| 07:49:29 | bauzas | okay, lemme look at the commit SHA1 | |
| 07:49:40 | gibi | that is the offending patch according to the reporter | |
| 07:50:29 | gibi | bauzas: I lean towards that the reporter do see a numa live migration regression on shared storage but I'm not sure that the root cause is the power mgmt flag added. But more like that flag now trigger a driver cleanup call that wasn't triggered before | |
| 07:50:49 | gibi | and that cleanup call might have issues | |
| 07:50:58 | bauzas | agreed | |
| 07:51:02 | bauzas | that's the cleanup | |
| 07:53:01 | bauzas | gibi: oh, that's the conditional | |
| 07:53:16 | bauzas | we only cleanup if we don't have shared path | |
| 07:53:28 | bauzas | but here, the 'or' makes it now optional | |
| 07:53:47 | bauzas | we just need to change the conditional | |
| 07:56:04 | gibi | calling the cleanup with destroy_disks should be OK | |
| 07:56:17 | gibi | destroy_disk= false | |
| 07:57:23 | gibi | appreantly it isn't OK but changing the logic at when to trigger do_cleanup seems to be the wrong fix to me | |
| 07:57:45 | gibi | as destroy_disk=false should prevent the disk deletion | |
| 07:57:54 | bauzas | true | |
| 07:58:19 | bauzas | but we check two different fields for shared storage | |
| 07:58:34 | bauzas | the conditional checks whether both computes use the same path | |
| 07:58:36 | gibi | yes I spotted that but did not dig deeper tehre | |
| 07:59:05 | bauzas | while the destroy_disks arg checks whether we use block storage | |
| 07:59:38 | bauzas | I assume the reporter gets something like True for the shared path but False for shared block storage | |
| 08:01:36 | gibi | what is a shared block storage? | |
| 08:01:41 | bauzas | for mdevs and vpmems, there was a reason to call cleanup | |
| 08:01:45 | gibi | just cinder? or something more special? | |
| 08:01:55 | bauzas | we need to delete the local residues | |
| 08:02:24 | bauzas | honestly, that's the first time I'm hearing this | |
| 08:02:45 | bauzas | I'm used to the regular FS share between instance pathes | |
| 08:03:01 | bauzas | that's the regular shared storage I know | |
| 08:03:28 | gibi | yeah I think is_shared_instance_path is the case when the instance dir is on e.g. NFS | |
| 08:03:41 | bauzas | yup | |
| 08:04:24 | gibi | we need to call cleanup if power mgmt is enabled so the current fix proposed would regress that | |
| 08:04:38 | bauzas | I need to see what makes migrate_data.is_shared_block_storage | |
| 08:04:54 | gibi | yep | |
| 08:05:02 | bauzas | and it would also regress the mdev live-migration support | |