Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-07-07
19:39:01 sean-k-mooney i have used them before
19:39:04 mikal Building N cloud images is actually a fairly big hosting problem I imagine. That's a fair bit of repeated data just for a very minor problem.
19:39:07 sean-k-mooney but it depend on yoru hardware
19:39:09 fungi seems like you'd want a more full-featured image with broader hardware support?
19:39:16 fungi yeah
19:39:18 sean-k-mooney somethime they dont have the drivers requried soemtimes they do
19:39:48 sean-k-mooney its been a long time since i tried but i think i used diskimage builder with the cloud image as a base to build one
19:40:19 sean-k-mooney cloud image doen nto mean qemu/kvm or vm based
19:40:21 fungi debian not including ahci support in the generic cloud kernels because "most" cloud providers (amazon/azure/google) is a good example of why i'd be skeptical of using such images in ironic for bare metal
19:40:31 mikal DIB definitely uses the various cloud images as the base for builds.
19:40:48 sean-k-mooney mikal: it can/does btu it can replace the kernel ectra
19:41:11 mikal sean-k-mooney: yeah, package installs etc etc.
19:41:23 fungi mmm, these days the dib elements we're using start from docker images instead, though previously we used elements that bootstrapped from scratch with tools like debootstrap
19:41:28 mikal This is how the SF "official" images are built. DIB with the SF agent sprinkled in.'
19:42:07 sean-k-mooney fungi: when did we move the ubuntu ones
19:42:09 fungi but yes, i think dib does have some non-"minimal" elements that instead modify cloud images with virsh tools
19:42:17 sean-k-mooney because they used to use the cloud image until relitivly recently
19:42:39 fungi sean-k-mooney: move from using debootstrap to container images? not sure if we have done that for the ubuntu images yet, i have to check
19:43:14 fungi but at least for the dib-created images we boot in opendev we've never used cloud images as the base
19:43:16 sean-k-mooney fungi: last time i looked and when i was lookign to add alpine most still did not use contianers
19:43:21 sean-k-mooney fedora i think did
19:43:32 sean-k-mooney but most still default to clodu images as a base
19:44:15 sean-k-mooney https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/ubuntu/root.d/10-cache-ubuntu-tarball#L16-L21
19:44:25 sean-k-mooney ok they use the sqsfs rather then the qcow
19:44:53 sean-k-mooney or rather the filesystem image
19:44:56 fungi sean-k-mooney: yeah, looks like we're still using ubuntu-minimal (debootstrap into a chroot) not ubuntu-container
19:45:28 sean-k-mooney so i know there is a fedora container element https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/elements/fedora-container
19:45:33 fungi the rocky images we build are using the rocky-container element by contrast
19:45:42 sean-k-mooney right
19:45:54 sean-k-mooney but it think that was only done for 1 or two and most didnt move yet
19:46:17 fungi yeah, looking back at it i think you're right, i assumed we were further along on that already
19:46:45 fungi i guess it's been on hold while we switch from nodepool to zuul-built images
19:47:35 sean-k-mooney i was playing with packer and trying to buidl a alpine image at the weekend and i was debating if i wanted to revive my dib serise for that
19:48:03 sean-k-mooney building os images is alwasy a pain regardess of the tooling
19:48:27 sean-k-mooney alpine now provides a cloud iamge with cloudinit however
19:48:39 sean-k-mooney its just larger then i was hoping
19:49:34 sean-k-mooney fungi: i still think exporeing alpine to have a replacement for cirros or moving to using nested kvm might be the only way we elimiate the kernel panics we see in our jobs
19:50:06 fungi it seems worth exploring, i agree
19:50:13 sean-k-mooney it does not have to be alpine but there are very few other distos that are small enough and well supproted to consider
19:51:24 sean-k-mooney anyway back to the topic at hand if folks are building an ironic image they can still use https://github.com/openstack/diskimage-builder/tree/master/diskimage_builder/elements/cloud-init-datasources#environment-variables
19:51:30 sean-k-mooney to list openstack
19:52:03 fungi i want to say someone (doug?) was playing with using distros that target embedded systems (openwrt? emdebian? building something with yocto?)
19:52:15 fungi but it's been years ago now
19:52:37 sean-k-mooney fungi: i got it partly work i was just missing growpart :)
19:53:08 sean-k-mooney but ya i brefily considerd tinycore since ironic used it for the ipa
19:54:22 sean-k-mooney the problem is the combination of small disk footpring <100MB and small ram foot pring <128MB + cloud-init or glean +busybox/tools needed byt tempest
19:55:38 sean-k-mooney fungi: now that they offically publish cloud images i shoudl proably jsut poing my testing patch at that and see if it works
19:56:01 fungi worth a shot, sure
19:57:30 sean-k-mooney fungi: how is the ci since the roolout of the zuul change
19:57:57 sean-k-mooney fungi: have cross provider builds been avoided as expected?
19:58:16 fungi sean-k-mooney: not 100% solved, we discovered that we also need https://review.opendev.org/954284
19:59:19 sean-k-mooney hum the cross tenant sharing of python object is slightly scary but ok
19:59:27 sean-k-mooney i guess that will rool out next weekend
19:59:49 fungi we'll probably restart the launchers onto it as soon as the container images promote
20:00:23 fungi turns out we had done that when the previous (partial) fix merged but i didn't realize it when discussing later in the week
20:03:03 sean-k-mooney oh i see the sharing is not a generic thing just in that funtion
20:03:32 sean-k-mooney i was slighly concured that semapheroes or secretes ectra could be shared but this is not related to the job objects
20:04:45 fungi no, you could look at it as the problem arising from a generally cross-tenant resource not having been correctly shared since zuul normally separates objects in one tenant from the others
20:06:49 opendevreview sean mooney proposed openstack/nova master: [WIP] use alpine instead of cirros https://review.opendev.org/c/openstack/nova/+/881912
20:08:46 sean-k-mooney i think i had previously customised the alpin image i built to allow password login with gocubsgo for tempest although that might works via ssh key loging
20:11:10 fungi amusing, i never realized that was the default password for cirros images. i guess someone's a baseball fan
20:11:15 sean-k-mooney its hard to tell becase i never wrote the nodepool change to build the image in ci i only built it locally so i dont recall if i use the dev-user element or not
20:11:44 mikal fungi: that would be Scott Moser.
20:11:49 sean-k-mooney it used to be "cubswin:)" until they one and now its "gocubsgo"
20:11:54 fungi hah
20:12:12 clarkb re dhcp vs metadata service I think a difference is that typically dhcp is a fairly locked down service within hosting providers. The hosting services may not all realize hey needt o prevent traffic on that special metadata address
20:12:26 clarkb basically lock down dhcp is pretty standard stuff. Lock down metadata service isn't
20:12:47 mikal I just think this is a pretty big breaking change for a relatively minor "security vulnerability".
20:12:48 sean-k-mooney im not sure. i woudl cosnider both to be simialr leveles of treat
20:13:02 clarkb sean-k-mooney: they are. But one is super standard that everyone knows about. The other is not
20:13:13 clarkb basically the differencesi s one has an rfc the other doesn't
20:13:19 mikal Especially when I can think of use cases which want to get rid of the customized DMI data entirely
20:13:48 fungi also i don't know how i would go about using a malicious dhcp server to get a machine to run arbitrary code at boot or install access for my personal ssh key
20:13:54 sean-k-mooney mikal: i mentione dthis else whre but the dmi info is technically configuable by pakcager/deployers
20:14:02 sean-k-mooney nova reads a vendor.conf file if it exits
20:14:07 sean-k-mooney which change what that info is
20:14:39 fungi but yeah, all of this is great feedback to the cloud-init maintainers
20:14:41 mikal So now deployers can accidentally break cloud-init with a DMI customization?
20:14:44 sean-k-mooney we do not use that downstream in redhat partly because we saw the ablity to tell it was openstack or worse what version of openstack as a potital security problem
20:14:45 mikal That's not great either.
20:14:55 fungi and now that the bug is public, i suppose comments could be added tehre
20:16:31 fungi sean-k-mooney: i don't see us relying on the dev-user element anywhere, we have a custom element that we inject ssh keys with
20:17:01 sean-k-mooney fungi: that was not upstream i ment whewn i built my test alpine image locally
20:17:08 fungi ah
20:17:34 sean-k-mooney wheni was workign on the dib support i was usign the dev-user image to debug but its been 4 years so i have no idea if the iamge i was testing with still had that or not
20:17:49 fungi okay, just re-read your earlier comment about dev-user and now i get what you were saying
20:18:21 sean-k-mooney fungi: i got it to boot in the ci and it apear to be workign but boots hung because i did not expand the disk
20:18:29 sean-k-mooney so it would start ot boot and then fail
20:19:44 sean-k-mooney i actully did 2 impelmetions the most recent one was 2 years ago based on teh continar file support https://review.opendev.org/c/openstack/diskimage-builder/+/755410/4
20:20:42 sean-k-mooney mikal: ya you need to have a file like this https://github.com/openstack/nova/blob/master/etc/nova/release.sample
20:20:53 sean-k-mooney mikal: https://github.com/openstack/nova/commit/4aa45d2900a3063305a7b2c727abfc2371909b21
20:21:27 sean-k-mooney the support for that still technially exists https://github.com/openstack/nova/blob/master/nova/version.py
20:21:37 sean-k-mooney but nothing i know of actully uses that
20:22:10 mikal But that's the key point. OpenStack tries hard not to break existing behaviour. cloud-init apparently does not.
20:22:39 mikal I think this is the strongest argument I've seen as to why people should move to glean, especially because glean implies that config drive will not in fact one day be removed.
20:23:14 sean-k-mooney there are other light wight alternitives too. the problem with glean is partly its streght
20:23:22 sean-k-mooney it does not supprot metadtaa only config drive
20:23:30 sean-k-mooney which si why glean is so reliable
20:24:15 sean-k-mooney but since we cant update config dirve it only really good for first boot but honestly if your doing day two thinbgs use ansible
20:25:04 fungi i don't think it's entirely fair to say that cloud-init doesn't try to preserve existing behaviors, just in this particular case they judged the existing behavior to be a greater risk than the impact from breaking it (whether they judged that accurately is another matter)

Earlier   Later