Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-09-06
13:44:56 sean-k-mooney i likely can make that happen
13:45:18 sean-k-mooney but using our existign create partion tbale and create partion functions didnt seam to be enough
13:45:52 dansmith can you show me fdisk -l of what we create?
13:46:25 sean-k-mooney sure ill need to recreate the image but ill chuck taht in termbin in a sec
13:49:02 sean-k-mooney https://termbin.com/86m6
13:49:29 sean-k-mooney i can get it to pass the first test_create_file but not test_cache_file
13:49:51 opendevreview Merged openstack/os-vif master: Update master for stable/2024.2 https://review.opendev.org/c/openstack/os-vif/+/928356
13:50:11 dansmith okay that doesn't show the mbr part because it skips right over, but must have a real pmbr
13:50:28 dansmith can you grab just the first like 4k of that and post it for me somewhere?
13:50:36 dansmith (if it's easy)
13:51:38 sean-k-mooney sure i have this running in a debugger adn im stop right after we hcage created the image but before we call mkfs on it
13:51:54 sean-k-mooney im wonder fi that will alter the output but ill grab the first 4k first and then proceed
13:52:16 dansmith hmm, how is mkfs going to run on just the partition of a file?
13:52:27 dansmith ...without kpartx or something
13:52:52 sean-k-mooney im actully expecting it to recreate teh gpt and partion table
13:52:59 dansmith mkfs will definitely not do that
13:53:14 sean-k-mooney ok then we might need to actuly loop mount it
13:53:17 dansmith if you run mkfs on a file it will create a filesystem starting at byte/sector 0 in the file, wiping the table
13:53:26 sean-k-mooney the current code calls mkfs direcly on the file
13:54:02 dansmith okay so maybe we're creating a partition table for the unformatted case, but if we format with a filesystem, we immediately blow that away and then hand the guest a whole-disk filesystem with no partition table?
13:54:03 sean-k-mooney ya so that might be the problem im coing to quickly copy that file and leave mkfs run
13:54:34 dansmith and to be clear, changing that is not easy
13:54:39 sean-k-mooney dansmith: so creating the partition tabel and partion was what i added to try and "fix" this
13:54:47 sean-k-mooney the exisign code just ran mkfs on the file
13:54:50 dansmith oh okay
13:55:00 sean-k-mooney or block device depending on the code path
13:55:24 dansmith but we were creating a fs in the file and that was being detected as GPT?
13:56:13 sean-k-mooney i can comment out my changes and verify i think it was detecated as a flatfile before but when used as a backinf file we expected something differnt
13:56:55 sean-k-mooney its proably better if i copy the files to a file share so you can look at it and also comment out the code i added to see the behavior before and after
13:57:23 dansmith and was this with vfat? because apparently vfat looks a lot like an MBR (unsurprisingly)
13:57:37 dansmith so maybe that inspector needs to look for something specific to not think fat is an MBR
13:58:26 sean-k-mooney ya so i chose vfat for the repoducer beacuse it is what we use if you dont specify a a format in the config and os_type is not set in the galnce image
13:58:58 sean-k-mooney if ostype is set in the glance iamge we will either use ntfs or ext4 but you have everal options for the config option
13:59:33 dansmith lol yep, vfat has a boot record and the same signature as an MBR in the first sector
13:59:39 dansmith thanks a lot 1980s MSFT
14:00:09 sean-k-mooney ok that might be the issue
14:00:16 dansmith "boot record" meaning 16-bit real-mode x86 executable code to show the message "this is not a bootable disk" :D
14:00:45 dansmith I have to run do something but when I get back I'll see what I can do to not be fooled into thinking a vfat is an MBR, but we'll see
14:01:24 dansmith still seems like we should be able to skip the backing file and skip the safety check on blank disks we create ourselves
14:01:28 sean-k-mooney well we have a few options but ya lets talk about it later
14:01:30 dansmith brb
14:02:13 sean-k-mooney yep i was also goign to explore that route by adding a flag to the image backend create_image to say if we should safty check it
14:02:37 sean-k-mooney but before doing that i wanted to see if i could avoid chaning the interface
14:04:41 sean-k-mooney https://fileshare.seanmooney.info/ephemeral_before_mkfs.img and https://fileshare.seanmooney.info/ephemeral_after_mkfs.img are teh relevent images, so the mkfs call undid what i was trying to do so that what nova produces today
14:05:22 sean-k-mooney this is the fdisk output https://termbin.com/bvvp
14:06:26 sean-k-mooney i renamed the files to .img and fixed the hardcoded extion im doing localy so ignore the fact it refecnce .qcow in the output
14:06:45 sean-k-mooney we alway use raw backing files for epmeral disks
14:07:09 sean-k-mooney well or ploop for openvz but we can ignore that
14:07:28 sean-k-mooney that proably broken too now but i dont think we care as much
14:08:43 sean-k-mooney we have not had any testing of openvz/Virtuozzo supprort for years so its unlikely that even still works without this
14:20:02 dansmith yeah, mkfs will definitely overwrite the partition table bits, the problem is vfat creates one again itself and adds a boot record (because it's actually a partition within a partition)
14:20:07 dansmith so this should not fail if you choose extN right?
14:24:00 sean-k-mooney ill check but i dont expect it to no, im just tryin gto figure out how to confirm that since the funcitonl test sute is complaining about privespe when i change the default
14:24:32 sean-k-mooney oh
14:24:37 sean-k-mooney no thats ok
14:24:42 sean-k-mooney it failed after the safty check
14:25:08 sean-k-mooney we call _update_utime_ignore_eacces(base) after the safty checks
14:26:55 sean-k-mooney so ya the inspecotr for ext4 returns RawFileInspecotr instead of the mbr/gpt one
14:27:56 sean-k-mooney dansmith: so i had a proposal to change our default to not format ephmeral disk
14:28:29 sean-k-mooney if i was to quickly implemnte that and wrap the mkfs call in an if then that woudl work for the default case
14:28:58 sean-k-mooney althernitivly i guess we coudl add a vfat inspector? or special case that in the mbr/gpt one
14:30:15 dansmith it's not that we need a vfat inspector, it's that we'd need the gpt one to not detect the vfat VBR as an MBR, or not obsess over the lack of partitions
14:30:23 dansmith or ideally, just not if it thinks it's a VBR instead of an MBR
14:30:24 sean-k-mooney i would prefer not to need to update oslo for this if we can avoid it but there are workaroudn that we can suggest to people in the interim
14:30:26 dansmith I'm looking into options
14:30:48 dansmith well, again, we shouldn't need to be inspecting blank images we're creating at all
14:30:53 dansmith so that's the non-oslo fix
14:31:45 sean-k-mooney right so i can implemant that with a new parmater to create_image in the image backend and default inspect_backingfile to true
14:31:55 sean-k-mooney then explcitly set it to false for epmeral disks
14:32:09 sean-k-mooney or ephmeral but only when vfat
14:32:46 dansmith all ephemeral should be fine I think
14:32:47 sean-k-mooney if we are ok with that approch ill stop trying ot make it pass and just swap to tuning it off for now when we are creating blank images
14:33:10 dansmith but yeah, that's really what we want to do I think and I'll work in parallel to stop the detection just in case anyone uses a raw vfat in a glance parition
14:33:19 dansmith *image
14:33:35 sean-k-mooney dansmith: are you ok with defering stoping creatin backing files for epmereal disks to the larger image backend refactor
14:34:22 dansmith certainly deferring it past the targeted fix for this
14:34:28 sean-k-mooney ack
14:34:43 dansmith and yeah, I think continuing to slice the current backend is just asking for trouble, even for something smallish like not using backing files I think
14:35:41 sean-k-mooney i feel like while i coudl make it work it woudl be more error prone espcialy when it compes to move ops and upgrade behavior
14:36:03 dansmith yeah
14:43:15 dansmith sean-k-mooney: are you okay with excluding images of floppy disks?
14:43:42 dansmith I don't mean a fake image you created to boot from floppy, but like an actual dd image of a floppy created on MS-DOS 6 or something
14:43:46 sean-k-mooney yes im also happy with droping floopy drive supprot form hw_disk_bus if we have not already
14:44:03 dansmith AFAIK, qemu dropped floppy support a while back anyway right?
14:44:17 sean-k-mooney it did in q35
14:44:23 dansmith ah okay
14:44:24 sean-k-mooney and i think we did downstream
14:44:30 sean-k-mooney btu not sure about upstream with pc
14:44:39 dansmith well, we could check a range, but I think it's easier if we just check for the descriptor that says "this was created on a hard disk"
14:53:52 dansmith sean-k-mooney: https://review.opendev.org/c/openstack/oslo.utils/+/928448
14:54:00 dansmith I will get a test image up too
14:56:39 sean-k-mooney im using oslo.utils form a local git repo so i can apply that and test it in my funcitonal test
14:57:43 dansmith cool
15:01:30 sean-k-mooney Switched to branch "review/dan_smith/cut-the-fat" :)
15:01:37 dansmith you're welcome :)
15:02:19 gibi lol
15:11:36 opendevreview Dan Smith proposed openstack/nova master: DNM: Test with FAT image https://review.opendev.org/c/openstack/nova/+/928451
15:11:58 dansmith sean-k-mooney: we'll make sure this ^ fails the same way you're seeing with ephemeral and then add in the oslo change to see that it cures
16:25:26 sean-k-mooney dansmith: sorry was on a call, so if i include your oslo fix and revert back to vfat it seams to be working. its failing at the same point where it tries to use privesep post the safty check
16:25:39 sean-k-mooney ill look a little closer now and confirm

Earlier   Later