Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-10-22
11:11:13 zigo Ok, but then, what our format_inspector is trying ot acheive is *not* checking if some images aren't respecting the GPT specs. We're trying to check if nova/cinder/glance is being hacked...
11:12:12 zigo I wouldn't mind fixing the Debian if it's broken, that's probably the case, but that wont fix the "I cannot migrate my VMs from my old cloud" use case...
11:12:31 zigo (and that's what I'm currently trying to fix)
11:12:32 sean-k-mooney we are tryign to require "valid" boot image and currently that includes a valid GPT partion table
11:12:51 sean-k-mooney let me see if i can find the perio bug
11:13:00 sean-k-mooney i have forgotten much of the context ath this point
11:13:30 zigo Right. We aren't trying to prevent border-line wrongly not respecting the spec images.
11:13:49 zigo Just something that Qemu can boot safely.
11:14:38 zigo It'd be ok to be more respecting the spec if we were writing a spec validation tool for qemu, but that's not our goal ...
11:14:47 sean-k-mooney https://bugs.launchpad.net/oslo.utils/+bug/2091114
11:15:03 zigo Thanks, reading.
14:36:01 Uggla Does someone knows about https://review.opendev.org/c/openstack/nova-specs/+/964444 ? I mean do we have the owners of this patch around ?
14:43:23 sean-k-mooney i just saw the propal come in yesterday
14:43:38 sean-k-mooney so i dont know anything more beyond what is in the review
15:30:47 zigo dansmith: I do not agree with your comment in the review. From your review:
15:30:47 zigo "Removing checks from a security layer because Debian's images have been wrong for a long time is not a good plan, IMHO."
15:30:47 zigo It is wrong that these magic byte checks are doing any security improvements. Plus I'm not sure where this image is comming from, I'm not sure it's from Debian, but it's been there in our older cloud that I'm trying to migrate.
15:31:17 dansmith zigo: understood :)
15:31:19 zigo IMO, we should re-open the topic during the vPTG.
15:31:49 zigo I do agree some kind of checking is good, and thanks for writing these.
15:32:34 zigo Though blocking existing (customer) VMs that's been there for so long isn't helpful.
15:32:56 zigo If I'm being ignored, I'll cary this as Debian specific patches ... :P
15:33:04 dansmith these things are 100% security related btw.. the configuration of the MBR is read and interpreted by the hypervisor, the emulated BIOS/firmware, etc.. so it's entirely possible there's an attack vector there
15:33:41 zigo At what stage the hypervisor reads it?!?
15:34:00 zigo Shouldn't this be done by the virtual BIOS when booting?
15:34:08 zigo (ie: not even in Qemu)
15:34:24 dansmith the hypervisor is running the virtual bios :)
15:34:40 zigo Well in qemu, no?
15:35:02 dansmith yes.. you know qemu is the thing that was hackable with the previous image attack right?
15:35:24 zigo If there was a way to escape when Qemu boots an HDD, that would be a serious security issue in Qemu.
15:35:43 dansmith escape is not the only attack mode, of course
15:35:45 zigo Well, the thing that was hackable, was the image conversion.
15:36:06 dansmith no it wasnt.. you might want to re-read the bug :)
15:36:11 zigo That's very different from executing sea-BIOS instructions.
15:36:15 dansmith it was literally qemu itself
15:37:25 opendevreview Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130
15:37:34 zigo When reading the bootloader stuff?!?
15:38:42 dansmith when reading the image and executing the guest.. not the bootloader specifically, but again, we're talking about being defensive for the *next* thing here
15:39:33 dansmith anyway, re-open the discussion of supporting spec-violating images in the vPTG if you want, and carry debian patches to remove these matches-the-spec image security checks if you think that's the best plan
15:40:19 zigo IMO, here, we're talking about being deffensive against the sea-BIOS being executed and doing weird thing, which is *not* our job, and which is hopefully harmless.
15:40:52 zigo But fine ... :P
15:41:07 dansmith well, I thought that inspecting the details of even things like qcow2 data structures was completely not our job until qemu told us it was
15:41:13 zigo BTW, my first iteration got things wrong, what I added wasn't the boot signature I've found.
15:41:28 zigo I've found (0x00, 0x01, 0x00)
15:42:02 zigo dansmith: Right. It probably deserves a discussion. :)
15:42:28 zigo Thanks for this begining of a chat already.
15:43:27 dansmith personally, I think we've already had the discussion, and we've already had a prior art bug that shows another distro realized they were wrong and posted the easy "make the image match the spec" commands, which we could put in the docs
15:44:34 zigo I probably can also patch the MBR of these old VMs...
15:44:43 zigo I'll see if that path works and let you know.
15:46:40 dansmith maybe glance could have a feature in image conversion to fix these things specifically with parted.. that's something I'd be on board with and probably would address your migration issue
15:50:56 sean-k-mooney zigo: (0x00, 0x01, 0x00) is because of a bug in parted
15:51:09 sean-k-mooney acording to the the flatcar image fix pr anyway
15:51:22 zigo Ah... thanks a lot! :)
15:51:26 zigo I'll look it up.
15:51:56 zigo FYI, the image I'm talking about is probably running Debian Jessie ... :)
15:52:13 sean-k-mooney https://github.com/flatcar/scripts/pull/2668#issuecomment-2660859901
15:53:54 sean-k-mooney i have no idea how the flatcar image buildign work by the way
15:54:46 sean-k-mooney they provided a workaroudn for old images https://bugs.launchpad.net/oslo.utils/+bug/2091114/comments/12
15:56:39 sean-k-mooney by the way on the glance side ther eis https://github.com/openstack/glance/blob/master/glance/common/config.py#L123C1-L138 to allow folks to weaken the security
15:56:57 sean-k-mooney but that is not really a good longterm approch and im not conviced that shoudl be in oslo
15:57:17 zigo Surprisingly, Glance isn't complaining, only Nova, not sure why.
15:57:20 dansmith can't (and shouldn't) be in oslo
15:58:16 kevko hi, i am running nova upgrade check FROM caracal container when all others containers (kolla-ansible deployment) are still antelope ... this should pass , shouldn't ? as it is slurp ...but check failed ... is it ok ? https://paste.openstack.org/show/bXbzW9LwWQwQIEu8vEUV/
15:58:17 dansmith glance hasn't had upload stream safety checks for very long, so depends on what you're upgrading to
15:58:19 sean-k-mooney dansmith: i was honesstly kind of surpised ot see that in glance if im honest i assuem this is planned to be deprecated and remvoed eventually
15:58:49 kevko point is that i will upgrade control plane to caracal ...and my 300 compute servers will be upgraded one by one during the weeks
15:58:54 dansmith sean-k-mooney: yes, I don't like it either
15:59:34 sean-k-mooney kevko: ya you shoudl be able to upgrade the contolers and do the compute later
16:00:25 sean-k-mooney kevko: that looks like you have not rant either the db migration or somethign similar to that
16:00:38 sean-k-mooney it could also be using the wrogn db
16:01:58 sean-k-mooney the compute_id was added by https://github.com/openstack/nova/commit/a47fdef1bfba8b1e2c3279f2c51eda3d8a985ec1
16:02:07 sean-k-mooney in bobcat
16:02:35 kevko yeah, but nova-status upgrade check command sounds like "this script will tell you if you can upgrade , which of course including db migration "
16:03:34 sean-k-mooney well your using a nova-status form carical but have not applied a db migration form bobcat
16:03:36 kevko sean-k-mooney: as I said ... antelope is slurp ...so upgrade to caracal and caracal's nova-status upgrade check should work no ?
16:04:05 sean-k-mooney i would need to check our doc for if the status check shoudl be done after db sync or not
16:04:13 kevko sean-k-mooney: yeah, but why I should update db migration from bobcat when antelope -> caracal is supported slurp upgrade ?
16:04:45 sean-k-mooney its not that you do them form bobcat it that you woudl run all the caracal ones
16:04:49 kevko that's the point ...
16:06:37 kevko sean-k-mooney: In my understanding of the world, a DB migration is part of the upgrade... which means the upgrade check should take into account that I’m running it before the upgrade, right? At least that makes more sense to me…
16:06:40 sean-k-mooney kevko: in gerade we run nova stats after you have done the db upgrade
16:06:42 sean-k-mooney https://opendev.org/openstack/grenade/src/branch/master/projects/60_nova/upgrade.sh#L104
16:07:11 sean-k-mooney keveko you shoudl run the antelop version before the dbsync and the caracal one after
16:07:28 opendevreview Arnaud Morin proposed openstack/nova master: Fix: add a default dev_type in spec https://review.opendev.org/c/openstack/nova/+/964552
16:08:38 sean-k-mooney kevko: https://docs.openstack.org/nova/latest/admin/upgrades.html#rolling-upgrade-process
16:08:44 sean-k-mooney that what we docuemtn as well by the way
16:09:14 sean-k-mooney the db sysnc syste the schema which will work with the old code
16:09:35 sean-k-mooney the upgade check expct the new db schema
16:12:15 kevko sean-k-mooney: hmm, thank you.... so upgrade: antelope container -> run upgrade check from antelope container -> exec db migrations to caracal -> replace antelope container to caracal -> run upgrade check again from caracal container
16:18:27 sean-k-mooney basiclly. the first execution fo the status check tells you if you have any online migration or other thing you need to do before upgrading
16:18:37 sean-k-mooney the second tell you the same after you have upgraded
16:24:34 opendevreview Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130
18:54:14 opendevreview Merged openstack/nova master: pci: Add more detail and examples to pci.alias docs https://review.opendev.org/c/openstack/nova/+/950659
21:47:13 atmark Hello, I have instances that automatically got shutdown on few computes after restarting nova
21:47:20 atmark `During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor.`
21:47:36 atmark Full logs https://paste.openstack.org/show/bZzH7XnMnZut2zZzTvuk/
21:51:10 atmark OpenStack Version is Yoga
21:52:40 atmark Which table in the db do I need to look at to identify more VMs that has power state mismatch?
#openstack-nova - 2025-10-23
08:15:43 opendevreview Arnaud Morin proposed openstack/nova master: Fix: avoid comparing dev_type if absent https://review.opendev.org/c/openstack/nova/+/964552
14:39:12 Uggla Info: https://etherpad.opendev.org/p/nova-2026.1-ptg is almost done. I still need to schedule a couple of topics, especially one in Japan friendly timeframe.

Earlier   Later