| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-08-30 | |||
| 21:50:12 | sean-k-mooney[m] | there is no reason your bfv sereise would not work with the lvm driver right | |
| 21:50:53 | sean-k-mooney[m] | i have a devstack deploying to test the user_data update but forgot to enable ceph | |
| 21:51:16 | sean-k-mooney[m] | but i can pull in your changes after and test them if i get time after | |
| 23:14:34 | sean-k-mooney[m] | dansmith: https://termbin.com/6502 | |
| 23:15:27 | sean-k-mooney[m] | ill try ubuntu 20.04 in the morning but on c9s the regenreation fails | |
| 23:15:33 | dansmith | ugh | |
| 23:15:47 | dansmith | I was not expecting that kind of failure | |
| 23:16:01 | dansmith | seems if we get to the mkisofs part we should be as good as otherwise | |
| 23:16:12 | sean-k-mooney[m] | not when we are just calling an exsiting funciton | |
| 23:16:27 | dansmith | so the configdrive was created properly on instance boot but failed on regenerate? | |
| 23:17:17 | sean-k-mooney[m] | ya it might be because of the format | |
| 23:17:28 | sean-k-mooney[m] | this might be trying to update the iso | |
| 23:18:11 | dansmith | oh you mean code that works if the iso doesn't exist but fails if it already does? | |
| 23:18:13 | sean-k-mooney[m] | when we use vfat that is allow but for iso it need to delete and recreate the file | |
| 23:18:21 | dansmith | right | |
| 23:18:39 | dansmith | I guess I expected it to overwrite, but maybe that's why permission denied, because qemu owns it now? | |
| 23:18:39 | sean-k-mooney[m] | ya the previous iff unly ran that code if the file did not exist | |
| 23:18:43 | dansmith | yeah | |
| 23:18:51 | sean-k-mooney[m] | https://review.opendev.org/c/openstack/nova/+/816157/15/nova/virt/libvirt/driver.py#4950 | |
| 23:19:01 | dansmith | man, if this really doesn't work (with isofs) and nobody tested until now ... :P | |
| 23:19:17 | sean-k-mooney[m] | the reboot continues | |
| 23:19:25 | sean-k-mooney[m] | the error is logged and ignored | |
| 23:19:41 | sean-k-mooney[m] | so if you dont actuly look at the logs it looks like it worked form the api | |
| 23:19:58 | dansmith | yeah, but I hope that's not why it went unnoticed | |
| 23:20:11 | dansmith | this is also why a tempest test would be good... | |
| 23:20:46 | dansmith | early in the bfv rebuild, we were doing the same.. quietly not doing the rebuild, and I wrote a test that touches a file, then does the rebuild, and asserts that it's gone.. and that pointed out that we weren't actually doing it | |
| 23:22:40 | sean-k-mooney[m] | lookign at the vfat path i think i twould repoen the file and reformat it https://github.com/openstack/nova/blob/master/nova/virt/configdrive.py#L100 | |
| 23:23:32 | sean-k-mooney[m] | although im kind fo surrpised its failing like this for iso | |
| 23:23:32 | sean-k-mooney[m] | the upper fucntion https://github.com/openstack/nova/blob/master/nova/virt/configdrive.py#L137 | |
| 23:23:54 | dansmith | I gotta run in a minute | |
| 23:23:59 | sean-k-mooney[m] | oh never mind | |
| 23:23:59 | dansmith | I thought you were going to do this tomorrow? | |
| 23:24:03 | sean-k-mooney[m] | yep | |
| 23:24:30 | sean-k-mooney[m] | so the uper function creates a temp dir with teh metadata files and then turns that into an iso using the final pat as the output | |
| 23:24:42 | sean-k-mooney[m] | i tought it was going to do it in the temp dir and move it | |
| 23:25:01 | sean-k-mooney[m] | so ya if qemu has a lock on that file it will cause a permission denined | |
| 23:25:14 | dansmith | well, I think libvirt chowns it before boot doesn't it? | |
| 23:25:18 | dansmith | I was thinking ownership not lock | |
| 23:27:37 | sean-k-mooney[m] | the issue is this is happening at the wrong time | |
| 23:27:40 | sean-k-mooney[m] | the vm is still runing | |
| 23:27:48 | sean-k-mooney[m] | it need to happen when the vm is stopped | |
| 23:27:50 | dansmith | oh right, before the destroyed message | |
| 23:28:21 | dansmith | if this really needs that level of care, I'm going to recommend we swap the order of the bfv stuff | |
| 23:28:24 | sean-k-mooney[m] | right now its at the top of hard reboot https://review.opendev.org/c/openstack/nova/+/816157/15/nova/virt/libvirt/driver.py#3887 | |
| 23:28:48 | dansmith | yeah, before destroy | |
| 23:29:16 | sean-k-mooney[m] | ya so i dont object to swaping the order given the issues with the user data patch as is | |
| 23:29:34 | sean-k-mooney[m] | anyway time for me to go sleep o/ | |
| 23:29:43 | dansmith | I guess if we're about to destroy, it's not *as* bad that we write to it before, but we could be competing with writes from the guest in the vfat case | |
| 23:29:48 | dansmith | still, wrong as you note | |
| 23:29:57 | dansmith | ack, thanks for testing, g'nite | |
| 23:32:45 | sean-k-mooney[m] | by the way i tested the nova-client and osc change too which do work altough im going to ask for the osc change to take a user-data file instead of a sting or atleast have that as an option | |
| 23:32:53 | sean-k-mooney[m] | im pretty sure thats what we do on boot | |
| 23:33:45 | sean-k-mooney[m] | passing the bases64 encoded sting on the command line is annowing without doing $(echo "stuff" | base64) | |
| 23:34:31 | sean-k-mooney[m] | i guess thats less important if the nova part is broken | |
| 23:35:06 | dansmith | oh yeah, I hadn't even looked | |
| 23:35:11 | dansmith | passed as a CLI string makes no sense | |
| #openstack-nova - 2022-08-31 | |||
| 06:24:54 | opendevreview | Rajat Dhasmana proposed openstack/python-novaclient master: Add support to rebuild boot volume 2.94 https://review.opendev.org/c/openstack/python-novaclient/+/827163 | |
| 07:05:41 | bauzas | good morning Nova | |
| 07:22:43 | Uggla | good morning | |
| 07:44:10 | bauzas | mmm, I forgot to ask about your preferences for that *yet again* virtual PTG | |
| 07:44:24 | bauzas | I mean, how many slots we should book | |
| 07:44:55 | bauzas | I'm about to book 4 x 4 slots (Tues-Fri) but I'll cancel some if we agree | |
| 07:49:25 | bauzas | just done, see the scheduled tracks https://ptg.opendev.org/ptg.html | |
| 07:49:30 | bauzas | I can unbook the slots | |
| 07:50:34 | gibi | morning | |
| 07:50:40 | gibi | bauzas: 4x4 works for me | |
| 07:50:58 | gibi | sean-k-mooney: thanks for testing the user data out. | |
| 07:51:05 | bauzas | gibi: we'll see how much we need once we have an agenda | |
| 07:51:14 | gibi | bauzas: sure | |
| 07:51:59 | gibi | sean-k-mooney, dansmith: in the light of the actual regeneration issue I agree to punt the user_data and land rebuild bfv first | |
| 07:52:47 | sean-k-mooney[m] | im locally trying to fix the patch by the way as a followup patch | |
| 08:03:32 | sean-k-mooney[m] | ok what i was trying wont work but i know what to do instead so trying that now | |
| 08:34:34 | hkominos | Hi guys. Quick question. When overriding policies for nova, Should one redefine the whole policy.json file or Can I just define the single policy that I want to override?My current policy file is empty {} | |
| 08:37:03 | bauzas | hkominos: no, you should just update what you need* | |
| 08:37:19 | gibi | hkominos: you only need to add to the file that you want to override | |
| 08:37:32 | gibi | bauzas: you won :) | |
| 08:37:37 | bauzas | \o/ | |
| 08:40:30 | hkominos | thanks you guys ! | |
| 08:40:58 | bauzas | gmann: good spot for the WIP status on my election patch | |
| 08:41:31 | bauzas | gmann: that's because I flagged the WIP status directly in Gerrit when I created the PS1 in Aug | |
| 08:41:45 | bauzas | now the patch is in active state | |
| 09:00:39 | sean-k-mooney[m] | gibi: this is such a can of worms | |
| 09:01:40 | sean-k-mooney[m] | i think to make this work we need to delete the old config drive first but that means we neeed to deal with all the image backends seperately since they dont have a rm() function | |
| 09:02:48 | bauzas | sean-k-mooney: a tl:dr about the problem ? | |
| 09:02:50 | sean-k-mooney[m] | i can add one but i was hoping this would be simple and then i rememebred this should also work for lvm and rbd so i cant just delete the file | |
| 09:03:22 | sean-k-mooney[m] | bauzas: the user data update feature is trying to rebuild the config deive on hard reboot if you updated the user data | |
| 09:03:30 | sean-k-mooney[m] | btu that fails with a permission denied\ | |
| 09:03:53 | sean-k-mooney[m] | i belive that is because the disk is owned by qemu | |
| 09:04:47 | sean-k-mooney[m] | so nova cant updated it and this need to work for rbd and lvm anyway so i cant really just assume its a file on disk and chown it | |
| 09:05:16 | sean-k-mooney[m] | the user_data feature is blocking the BFV rebuild feature because of the micorversion order | |
| 09:05:35 | sean-k-mooney[m] | so i was trying to unblock that but we are just going to have to change the order | |
| 09:06:21 | sean-k-mooney[m] | make BFV rebuild 2.93 and likely punt mutable user_data to AA | |
| 09:06:53 | sean-k-mooney[m] | there are other issue with rpc/testing in the userdata patch that likely wont get adressed without a FFE | |
| 09:07:17 | bauzas | agreed on flipping the microversions | |
| 09:07:33 | bauzas | at least for not blocking bfv rebuild | |
| 09:07:46 | gibi | sean-k-mooney[m]: do we store the config drive in lvm / rbd? I thought it is always a file on disk | |
| 09:08:03 | gibi | but agree if this is complicate then punt it | |
| 09:08:07 | bauzas | for userdata, we can't just tell "sorry you can't update your data because config drive" as this is a config-driven API behaviour | |
| 09:08:40 | bauzas | could we have a separate flag that would say "I allow you to update your userdata" and leave the responsibility to the ops to enable it ? | |
| 09:08:51 | kashyap | gibi: I think it's a file on the disk, too, the config-drive | |