Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-08-30
20:25:37 dansmith yeah the ordering was unfortunate
20:26:05 sean-k-mooney[m] i should have reviewed it more closly sorry. thanks for reviewing though even if the timing is not idea its better to get this right
20:27:24 dansmith yep, so maybe if you test in the morning and +W it we can get the others in the queue and I'll start on the proper RPC stuff based on my pastebin above when I'm around
21:23:35 opendevreview Rajat Dhasmana proposed openstack/python-novaclient master: Add support to rebuild boot volume https://review.opendev.org/c/openstack/python-novaclient/+/827163
21:31:59 opendevreview Rajat Dhasmana proposed openstack/python-novaclient master: Add support to rebuild boot volume 2.94 https://review.opendev.org/c/openstack/python-novaclient/+/827163
21:33:02 whoami-rajat ^ rebased on top of 2.93 to avoid conflicts later
21:34:47 whoami-rajat sean-k-mooney[m], jfyi, you've a -2 on client patch of 2.93 https://review.opendev.org/c/openstack/python-novaclient/+/816158
21:41:37 sean-k-mooney[m] oh i didnt clear that after you spun
21:41:50 sean-k-mooney[m] i proably wont review that tonight but ill drop the -2 sorry about that
21:43:01 sean-k-mooney[m] oh thats the user data one
21:44:25 whoami-rajat yep, you already dropped the -2 from mine some time back but without the user data one, my change can't get in :)
21:48:05 sean-k-mooney[m] i ment to do that when the spec was approved
21:50:12 sean-k-mooney[m] there is no reason your bfv sereise would not work with the lvm driver right
21:50:53 sean-k-mooney[m] i have a devstack deploying to test the user_data update but forgot to enable ceph
21:51:16 sean-k-mooney[m] but i can pull in your changes after and test them if i get time after
23:14:34 sean-k-mooney[m] dansmith: https://termbin.com/6502
23:15:27 sean-k-mooney[m] ill try ubuntu 20.04 in the morning but on c9s the regenreation fails
23:15:33 dansmith ugh
23:15:47 dansmith I was not expecting that kind of failure
23:16:01 dansmith seems if we get to the mkisofs part we should be as good as otherwise
23:16:12 sean-k-mooney[m] not when we are just calling an exsiting funciton
23:16:27 dansmith so the configdrive was created properly on instance boot but failed on regenerate?
23:17:17 sean-k-mooney[m] ya it might be because of the format
23:17:28 sean-k-mooney[m] this might be trying to update the iso
23:18:11 dansmith oh you mean code that works if the iso doesn't exist but fails if it already does?
23:18:13 sean-k-mooney[m] when we use vfat that is allow but for iso it need to delete and recreate the file
23:18:21 dansmith right
23:18:39 sean-k-mooney[m] ya the previous iff unly ran that code if the file did not exist
23:18:39 dansmith I guess I expected it to overwrite, but maybe that's why permission denied, because qemu owns it now?
23:18:43 dansmith yeah
23:18:51 sean-k-mooney[m] https://review.opendev.org/c/openstack/nova/+/816157/15/nova/virt/libvirt/driver.py#4950
23:19:01 dansmith man, if this really doesn't work (with isofs) and nobody tested until now ... :P
23:19:17 sean-k-mooney[m] the reboot continues
23:19:25 sean-k-mooney[m] the error is logged and ignored
23:19:41 sean-k-mooney[m] so if you dont actuly look at the logs it looks like it worked form the api
23:19:58 dansmith yeah, but I hope that's not why it went unnoticed
23:20:11 dansmith this is also why a tempest test would be good...
23:20:46 dansmith early in the bfv rebuild, we were doing the same.. quietly not doing the rebuild, and I wrote a test that touches a file, then does the rebuild, and asserts that it's gone.. and that pointed out that we weren't actually doing it
23:22:40 sean-k-mooney[m] lookign at the vfat path i think i twould repoen the file and reformat it https://github.com/openstack/nova/blob/master/nova/virt/configdrive.py#L100
23:23:32 sean-k-mooney[m] the upper fucntion https://github.com/openstack/nova/blob/master/nova/virt/configdrive.py#L137
23:23:32 sean-k-mooney[m] although im kind fo surrpised its failing like this for iso
23:23:54 dansmith I gotta run in a minute
23:23:59 dansmith I thought you were going to do this tomorrow?
23:23:59 sean-k-mooney[m] oh never mind
23:24:03 sean-k-mooney[m] yep
23:24:30 sean-k-mooney[m] so the uper function creates a temp dir with teh metadata files and then turns that into an iso using the final pat as the output
23:24:42 sean-k-mooney[m] i tought it was going to do it in the temp dir and move it
23:25:01 sean-k-mooney[m] so ya if qemu has a lock on that file it will cause a permission denined
23:25:14 dansmith well, I think libvirt chowns it before boot doesn't it?
23:25:18 dansmith I was thinking ownership not lock
23:27:37 sean-k-mooney[m] the issue is this is happening at the wrong time
23:27:40 sean-k-mooney[m] the vm is still runing
23:27:48 sean-k-mooney[m] it need to happen when the vm is stopped
23:27:50 dansmith oh right, before the destroyed message
23:28:21 dansmith if this really needs that level of care, I'm going to recommend we swap the order of the bfv stuff
23:28:24 sean-k-mooney[m] right now its at the top of hard reboot https://review.opendev.org/c/openstack/nova/+/816157/15/nova/virt/libvirt/driver.py#3887
23:28:48 dansmith yeah, before destroy
23:29:16 sean-k-mooney[m] ya so i dont object to swaping the order given the issues with the user data patch as is
23:29:34 sean-k-mooney[m] anyway time for me to go sleep o/
23:29:43 dansmith I guess if we're about to destroy, it's not *as* bad that we write to it before, but we could be competing with writes from the guest in the vfat case
23:29:48 dansmith still, wrong as you note
23:29:57 dansmith ack, thanks for testing, g'nite
23:32:45 sean-k-mooney[m] by the way i tested the nova-client and osc change too which do work altough im going to ask for the osc change to take a user-data file instead of a sting or atleast have that as an option
23:32:53 sean-k-mooney[m] im pretty sure thats what we do on boot
23:33:45 sean-k-mooney[m] passing the bases64 encoded sting on the command line is annowing without doing $(echo "stuff" | base64)
23:34:31 sean-k-mooney[m] i guess thats less important if the nova part is broken
23:35:06 dansmith oh yeah, I hadn't even looked
23:35:11 dansmith passed as a CLI string makes no sense
#openstack-nova - 2022-08-31
06:24:54 opendevreview Rajat Dhasmana proposed openstack/python-novaclient master: Add support to rebuild boot volume 2.94 https://review.opendev.org/c/openstack/python-novaclient/+/827163
07:05:41 bauzas good morning Nova
07:22:43 Uggla good morning
07:44:10 bauzas mmm, I forgot to ask about your preferences for that *yet again* virtual PTG
07:44:24 bauzas I mean, how many slots we should book
07:44:55 bauzas I'm about to book 4 x 4 slots (Tues-Fri) but I'll cancel some if we agree
07:49:25 bauzas just done, see the scheduled tracks https://ptg.opendev.org/ptg.html
07:49:30 bauzas I can unbook the slots
07:50:34 gibi morning
07:50:40 gibi bauzas: 4x4 works for me
07:50:58 gibi sean-k-mooney: thanks for testing the user data out.
07:51:05 bauzas gibi: we'll see how much we need once we have an agenda
07:51:14 gibi bauzas: sure
07:51:59 gibi sean-k-mooney, dansmith: in the light of the actual regeneration issue I agree to punt the user_data and land rebuild bfv first
07:52:47 sean-k-mooney[m] im locally trying to fix the patch by the way as a followup patch
08:03:32 sean-k-mooney[m] ok what i was trying wont work but i know what to do instead so trying that now
08:34:34 hkominos Hi guys. Quick question. When overriding policies for nova, Should one redefine the whole policy.json file or Can I just define the single policy that I want to override?My current policy file is empty {}
08:37:03 bauzas hkominos: no, you should just update what you need*
08:37:19 gibi hkominos: you only need to add to the file that you want to override
08:37:32 gibi bauzas: you won :)
08:37:37 bauzas \o/
08:40:30 hkominos thanks you guys !
08:40:58 bauzas gmann: good spot for the WIP status on my election patch
08:41:31 bauzas gmann: that's because I flagged the WIP status directly in Gerrit when I created the PS1 in Aug
08:41:45 bauzas now the patch is in active state
09:00:39 sean-k-mooney[m] gibi: this is such a can of worms
09:01:40 sean-k-mooney[m] i think to make this work we need to delete the old config drive first but that means we neeed to deal with all the image backends seperately since they dont have a rm() function
09:02:48 bauzas sean-k-mooney: a tl:dr about the problem ?
09:02:50 sean-k-mooney[m] i can add one but i was hoping this would be simple and then i rememebred this should also work for lvm and rbd so i cant just delete the file
09:03:22 sean-k-mooney[m] bauzas: the user data update feature is trying to rebuild the config deive on hard reboot if you updated the user data
09:03:30 sean-k-mooney[m] btu that fails with a permission denied\
09:03:53 sean-k-mooney[m] i belive that is because the disk is owned by qemu

Earlier   Later