| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-09-06 | |||
| 13:08:25 | opendevreview | OpenStack Release Bot proposed openstack/os-vif stable/2024.2: Update TOX_CONSTRAINTS_FILE for stable/2024.2 https://review.opendev.org/c/openstack/os-vif/+/928355 | |
| 13:08:27 | opendevreview | OpenStack Release Bot proposed openstack/os-vif master: Update master for stable/2024.2 https://review.opendev.org/c/openstack/os-vif/+/928356 | |
| 13:08:31 | opendevreview | OpenStack Release Bot proposed openstack/osc-placement stable/2024.2: Update .gitreview for stable/2024.2 https://review.opendev.org/c/openstack/osc-placement/+/928358 | |
| 13:08:33 | opendevreview | OpenStack Release Bot proposed openstack/osc-placement stable/2024.2: Update TOX_CONSTRAINTS_FILE for stable/2024.2 https://review.opendev.org/c/openstack/osc-placement/+/928359 | |
| 13:08:34 | opendevreview | OpenStack Release Bot proposed openstack/osc-placement master: Update master for stable/2024.2 https://review.opendev.org/c/openstack/osc-placement/+/928360 | |
| 13:08:42 | opendevreview | OpenStack Release Bot proposed openstack/python-novaclient stable/2024.2: Update .gitreview for stable/2024.2 https://review.opendev.org/c/openstack/python-novaclient/+/928363 | |
| 13:08:43 | opendevreview | OpenStack Release Bot proposed openstack/python-novaclient stable/2024.2: Update TOX_CONSTRAINTS_FILE for stable/2024.2 https://review.opendev.org/c/openstack/python-novaclient/+/928364 | |
| 13:08:44 | opendevreview | OpenStack Release Bot proposed openstack/python-novaclient master: Update master for stable/2024.2 https://review.opendev.org/c/openstack/python-novaclient/+/928366 | |
| 13:38:45 | sean-k-mooney | dansmith: so i tought i had a fix for https://review.opendev.org/c/openstack/nova/+/928310/1/nova/tests/functional/regressions/test_bug_backing_file_partion_tables.py | |
| 13:39:05 | sean-k-mooney | and then whe i treid it i reasised that its more complex then i hoped | |
| 13:39:57 | sean-k-mooney | tl;dr is the mbr/gpt inspector is not only looking for a gpt partion table to exist its also looking for a os partion | |
| 13:40:22 | sean-k-mooney | for epmeral and swap disk that is not really a valid expecatoin | |
| 13:40:56 | sean-k-mooney | so just addign a gpt tabel and vfat partion is not enough to pass the safty check | |
| 13:42:17 | dansmith | It's not looking at the gpt part at all, | |
| 13:42:26 | dansmith | it's looking at the PMBR to make sure you have the right single entry | |
| 13:42:59 | sean-k-mooney | yes its looking at the mbr part | |
| 13:44:15 | dansmith | the mbr needs to have at least one partition that is not type EE, or a single type EE starting at the first sector | |
| 13:44:56 | sean-k-mooney | i likely can make that happen | |
| 13:45:18 | sean-k-mooney | but using our existign create partion tbale and create partion functions didnt seam to be enough | |
| 13:45:52 | dansmith | can you show me fdisk -l of what we create? | |
| 13:46:25 | sean-k-mooney | sure ill need to recreate the image but ill chuck taht in termbin in a sec | |
| 13:49:02 | sean-k-mooney | https://termbin.com/86m6 | |
| 13:49:29 | sean-k-mooney | i can get it to pass the first test_create_file but not test_cache_file | |
| 13:49:51 | opendevreview | Merged openstack/os-vif master: Update master for stable/2024.2 https://review.opendev.org/c/openstack/os-vif/+/928356 | |
| 13:50:11 | dansmith | okay that doesn't show the mbr part because it skips right over, but must have a real pmbr | |
| 13:50:28 | dansmith | can you grab just the first like 4k of that and post it for me somewhere? | |
| 13:50:36 | dansmith | (if it's easy) | |
| 13:51:38 | sean-k-mooney | sure i have this running in a debugger adn im stop right after we hcage created the image but before we call mkfs on it | |
| 13:51:54 | sean-k-mooney | im wonder fi that will alter the output but ill grab the first 4k first and then proceed | |
| 13:52:16 | dansmith | hmm, how is mkfs going to run on just the partition of a file? | |
| 13:52:27 | dansmith | ...without kpartx or something | |
| 13:52:52 | sean-k-mooney | im actully expecting it to recreate teh gpt and partion table | |
| 13:52:59 | dansmith | mkfs will definitely not do that | |
| 13:53:14 | sean-k-mooney | ok then we might need to actuly loop mount it | |
| 13:53:17 | dansmith | if you run mkfs on a file it will create a filesystem starting at byte/sector 0 in the file, wiping the table | |
| 13:53:26 | sean-k-mooney | the current code calls mkfs direcly on the file | |
| 13:54:02 | dansmith | okay so maybe we're creating a partition table for the unformatted case, but if we format with a filesystem, we immediately blow that away and then hand the guest a whole-disk filesystem with no partition table? | |
| 13:54:03 | sean-k-mooney | ya so that might be the problem im coing to quickly copy that file and leave mkfs run | |
| 13:54:34 | dansmith | and to be clear, changing that is not easy | |
| 13:54:39 | sean-k-mooney | dansmith: so creating the partition tabel and partion was what i added to try and "fix" this | |
| 13:54:47 | sean-k-mooney | the exisign code just ran mkfs on the file | |
| 13:54:50 | dansmith | oh okay | |
| 13:55:00 | sean-k-mooney | or block device depending on the code path | |
| 13:55:24 | dansmith | but we were creating a fs in the file and that was being detected as GPT? | |
| 13:56:13 | sean-k-mooney | i can comment out my changes and verify i think it was detecated as a flatfile before but when used as a backinf file we expected something differnt | |
| 13:56:55 | sean-k-mooney | its proably better if i copy the files to a file share so you can look at it and also comment out the code i added to see the behavior before and after | |
| 13:57:23 | dansmith | and was this with vfat? because apparently vfat looks a lot like an MBR (unsurprisingly) | |
| 13:57:37 | dansmith | so maybe that inspector needs to look for something specific to not think fat is an MBR | |
| 13:58:26 | sean-k-mooney | ya so i chose vfat for the repoducer beacuse it is what we use if you dont specify a a format in the config and os_type is not set in the galnce image | |
| 13:58:58 | sean-k-mooney | if ostype is set in the glance iamge we will either use ntfs or ext4 but you have everal options for the config option | |
| 13:59:33 | dansmith | lol yep, vfat has a boot record and the same signature as an MBR in the first sector | |
| 13:59:39 | dansmith | thanks a lot 1980s MSFT | |
| 14:00:09 | sean-k-mooney | ok that might be the issue | |
| 14:00:16 | dansmith | "boot record" meaning 16-bit real-mode x86 executable code to show the message "this is not a bootable disk" :D | |
| 14:00:45 | dansmith | I have to run do something but when I get back I'll see what I can do to not be fooled into thinking a vfat is an MBR, but we'll see | |
| 14:01:24 | dansmith | still seems like we should be able to skip the backing file and skip the safety check on blank disks we create ourselves | |
| 14:01:28 | sean-k-mooney | well we have a few options but ya lets talk about it later | |
| 14:01:30 | dansmith | brb | |
| 14:02:13 | sean-k-mooney | yep i was also goign to explore that route by adding a flag to the image backend create_image to say if we should safty check it | |
| 14:02:37 | sean-k-mooney | but before doing that i wanted to see if i could avoid chaning the interface | |
| 14:04:41 | sean-k-mooney | https://fileshare.seanmooney.info/ephemeral_before_mkfs.img and https://fileshare.seanmooney.info/ephemeral_after_mkfs.img are teh relevent images, so the mkfs call undid what i was trying to do so that what nova produces today | |
| 14:05:22 | sean-k-mooney | this is the fdisk output https://termbin.com/bvvp | |
| 14:06:26 | sean-k-mooney | i renamed the files to .img and fixed the hardcoded extion im doing localy so ignore the fact it refecnce .qcow in the output | |
| 14:06:45 | sean-k-mooney | we alway use raw backing files for epmeral disks | |
| 14:07:09 | sean-k-mooney | well or ploop for openvz but we can ignore that | |
| 14:07:28 | sean-k-mooney | that proably broken too now but i dont think we care as much | |
| 14:08:43 | sean-k-mooney | we have not had any testing of openvz/Virtuozzo supprort for years so its unlikely that even still works without this | |
| 14:20:02 | dansmith | yeah, mkfs will definitely overwrite the partition table bits, the problem is vfat creates one again itself and adds a boot record (because it's actually a partition within a partition) | |
| 14:20:07 | dansmith | so this should not fail if you choose extN right? | |
| 14:24:00 | sean-k-mooney | ill check but i dont expect it to no, im just tryin gto figure out how to confirm that since the funcitonl test sute is complaining about privespe when i change the default | |
| 14:24:32 | sean-k-mooney | oh | |
| 14:24:37 | sean-k-mooney | no thats ok | |
| 14:24:42 | sean-k-mooney | it failed after the safty check | |
| 14:25:08 | sean-k-mooney | we call _update_utime_ignore_eacces(base) after the safty checks | |
| 14:26:55 | sean-k-mooney | so ya the inspecotr for ext4 returns RawFileInspecotr instead of the mbr/gpt one | |
| 14:27:56 | sean-k-mooney | dansmith: so i had a proposal to change our default to not format ephmeral disk | |
| 14:28:29 | sean-k-mooney | if i was to quickly implemnte that and wrap the mkfs call in an if then that woudl work for the default case | |
| 14:28:58 | sean-k-mooney | althernitivly i guess we coudl add a vfat inspector? or special case that in the mbr/gpt one | |
| 14:30:15 | dansmith | it's not that we need a vfat inspector, it's that we'd need the gpt one to not detect the vfat VBR as an MBR, or not obsess over the lack of partitions | |
| 14:30:23 | dansmith | or ideally, just not if it thinks it's a VBR instead of an MBR | |
| 14:30:24 | sean-k-mooney | i would prefer not to need to update oslo for this if we can avoid it but there are workaroudn that we can suggest to people in the interim | |
| 14:30:26 | dansmith | I'm looking into options | |
| 14:30:48 | dansmith | well, again, we shouldn't need to be inspecting blank images we're creating at all | |
| 14:30:53 | dansmith | so that's the non-oslo fix | |
| 14:31:45 | sean-k-mooney | right so i can implemant that with a new parmater to create_image in the image backend and default inspect_backingfile to true | |
| 14:31:55 | sean-k-mooney | then explcitly set it to false for epmeral disks | |
| 14:32:09 | sean-k-mooney | or ephmeral but only when vfat | |
| 14:32:46 | dansmith | all ephemeral should be fine I think | |
| 14:32:47 | sean-k-mooney | if we are ok with that approch ill stop trying ot make it pass and just swap to tuning it off for now when we are creating blank images | |
| 14:33:10 | dansmith | but yeah, that's really what we want to do I think and I'll work in parallel to stop the detection just in case anyone uses a raw vfat in a glance parition | |
| 14:33:19 | dansmith | *image | |
| 14:33:35 | sean-k-mooney | dansmith: are you ok with defering stoping creatin backing files for epmereal disks to the larger image backend refactor | |
| 14:34:22 | dansmith | certainly deferring it past the targeted fix for this | |
| 14:34:28 | sean-k-mooney | ack | |
| 14:34:43 | dansmith | and yeah, I think continuing to slice the current backend is just asking for trouble, even for something smallish like not using backing files I think | |
| 14:35:41 | sean-k-mooney | i feel like while i coudl make it work it woudl be more error prone espcialy when it compes to move ops and upgrade behavior | |
| 14:36:03 | dansmith | yeah | |
| 14:43:15 | dansmith | sean-k-mooney: are you okay with excluding images of floppy disks? | |
| 14:43:42 | dansmith | I don't mean a fake image you created to boot from floppy, but like an actual dd image of a floppy created on MS-DOS 6 or something | |
| 14:43:46 | sean-k-mooney | yes im also happy with droping floopy drive supprot form hw_disk_bus if we have not already | |