Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2024-05-16
17:37:10 melwitt that would be super :)
17:37:18 bauzas no, unified limits is waaaay simplier that f**** nvidia kernel usage :)
17:38:04 melwitt haha! yeah, I guess if you can nvidia then you can do anything 😆
17:39:11 bauzas at least now I know more about cuda and numba :)
17:39:47 bauzas like the fact we need to use a specific cuda release related to the mdev type
17:40:23 bauzas (and the kernel nvidia version)
18:11:46 sean-k-mooney i mean its not surpsing the have spit there mdev by vdi vs comptue usage in there skus forever
#openstack-nova - 2024-05-17
02:25:47 opendevreview melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913
03:32:53 opendevreview melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913
04:05:34 opendevreview melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913
04:26:56 opendevreview melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913
07:41:22 opendevreview Robert Hoffmann proposed openstack/nova master: Fix backups hanging in creating state https://review.opendev.org/c/openstack/nova/+/919923
10:05:31 opendevreview Robert Hoffmann proposed openstack/nova master: Fix backups hanging in creating state https://review.opendev.org/c/openstack/nova/+/919923
13:13:30 opendevreview Mauricio Faria de Oliveira proposed openstack/nova master: libvirt: Filter out disabled guest CPU features in _get_cpu_feature_traits() https://review.opendev.org/c/openstack/nova/+/919949
14:10:22 opendevreview Andrew Bonney proposed openstack/nova master: [WIP] Set port dns_name when unshelving instances https://review.opendev.org/c/openstack/nova/+/919961
17:56:14 opendevreview sean mooney proposed openstack/nova master: [WIP] ensure correct cleanup of multi-attach volumes https://review.opendev.org/c/openstack/nova/+/916322
17:56:14 opendevreview sean mooney proposed openstack/nova master: add functional repoducer for bug 2048837 https://review.opendev.org/c/openstack/nova/+/919979
17:57:09 sean-k-mooney i spend the better part of 6 hours trying to get that repoduce to pass ^ and then i realsied the reason i could not get it to pass was my fix didnt work...
17:57:29 sean-k-mooney on the plus side i now have a repoducer for the orginal bug...
17:59:39 melwitt too real
18:05:19 sean-k-mooney melwitt: you dont happen to knwo if we stash if a volume is a multi attach volume in the bdms do you
18:07:03 melwitt sean-k-mooney: there isn't a dedicated field for it that I see. I dunno if it could possibly be in the connection_info though
18:08:18 sean-k-mooney ack no worries
18:09:20 sean-k-mooney it is in the conection info yes
18:09:54 sean-k-mooney actully we do cache that in the bdms right
18:10:21 melwitt yeah. sorry, connection_info is a field on the bdm. I was just saying there isn't a field that is only for multiattach
18:11:33 sean-k-mooney right so the simple way to fix the bug is to take a lock on the volume when delete the instnace
18:11:36 sean-k-mooney basiclly @utils.synchronized(f'disconnect_volume_{volume_id}')
18:12:32 sean-k-mooney but instead of doing that in the virt driver in _disconnect_volume
18:12:56 melwitt ok I see
18:12:57 sean-k-mooney i need to do that in _delete_instance in the compute manger
18:13:10 sean-k-mooney but i only need to do that for multi attach volumes
18:14:46 sean-k-mooney i was orginally tring to do the locking as fine grained as possible
18:15:17 sean-k-mooney but that wont work
18:15:25 melwitt oh yeah. so you're thinking you can check bdm.connection_info to see if it's multiattach?
18:15:56 sean-k-mooney ya
18:16:38 sean-k-mooney the other option is to basiclaly reuse or reinvent https://github.com/openstack/nova/blob/ab3ca1e2059f951b44d723c15a5065c6532540da/nova/virt/libvirt/volume/mount.py#L33
18:16:38 melwitt seems like that would work 🤔
18:17:00 sean-k-mooney the _HostMountStateManager basicaly impelmentes refernce counting for host mounts
18:17:18 sean-k-mooney but we dont use that for anything other the nfs
18:19:22 sean-k-mooney the problem is this logic https://github.com/openstack/nova/blob/ab3ca1e2059f951b44d723c15a5065c6532540da/nova/virt/libvirt/driver.py#L2045-L2065
18:19:36 sean-k-mooney if we have 2 vms on the same host using the same multi attach volume
18:19:46 sean-k-mooney and you delete both of them at the same time
18:19:57 melwitt ah ok
18:20:16 sean-k-mooney then because we are looking at the host's instance list until know if we should do the disconenct there is a race
18:21:18 sean-k-mooney so to fix it we either need a refence count and delete it when that hits 0 or we need to just seriase the delete
18:21:32 sean-k-mooney i think the latter is simpler and safer
18:21:54 melwitt I agree
18:23:01 sean-k-mooney ok now that i know someone else does not think this is an insane approch i think im going to call it a day
18:23:18 melwitt haha :)
18:23:54 sean-k-mooney the worst part was i didnt even thing the issue was with my fix. i was confived the issue was with my repoducer test for so long
18:24:38 melwitt I know the feeling 😝
19:26:11 opendevreview melanie witt proposed openstack/nova master: docs: Add more information about unified limits https://review.opendev.org/c/openstack/nova/+/919913
20:51:19 opendevreview melanie witt proposed openstack/nova master: Consolidate vTPM and ephemeral encryption secret creation https://review.opendev.org/c/openstack/nova/+/912094
20:51:19 opendevreview melanie witt proposed openstack/nova master: Support create with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870932
20:51:20 opendevreview melanie witt proposed openstack/nova master: libvirt: Add optional 'description' kwarg to create_secret() https://review.opendev.org/c/openstack/nova/+/919986
23:42:27 opendevreview melanie witt proposed openstack/nova master: Support (resize|cold migration) with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870933
23:42:28 opendevreview melanie witt proposed openstack/nova master: Support live migration with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/905512
23:42:29 opendevreview melanie witt proposed openstack/nova master: Support rebuild with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/870939
23:42:31 opendevreview melanie witt proposed openstack/nova master: Add encryption support to qemu-img rebase https://review.opendev.org/c/openstack/nova/+/870936
23:42:31 opendevreview melanie witt proposed openstack/nova master: Add backing_encryption_secret_uuid to BlockDeviceMapping https://review.opendev.org/c/openstack/nova/+/907960
23:42:32 opendevreview melanie witt proposed openstack/nova master: Support encrypted backing files for qcow2 https://review.opendev.org/c/openstack/nova/+/907961
23:42:34 opendevreview melanie witt proposed openstack/nova master: Support rescue with ephemeral encryption for qcow2 https://review.opendev.org/c/openstack/nova/+/873675
23:42:34 opendevreview melanie witt proposed openstack/nova master: Validate key manager create access in API https://review.opendev.org/c/openstack/nova/+/919989
23:42:35 opendevreview melanie witt proposed openstack/nova master: Clean up unused ephemeral encryption libvirt secrets https://review.opendev.org/c/openstack/nova/+/919990
23:42:35 opendevreview melanie witt proposed openstack/nova master: libvirt: "Auto heal" ephemeral encryption secrets on guest launch https://review.opendev.org/c/openstack/nova/+/919991
23:42:36 opendevreview melanie witt proposed openstack/nova master: Support encrypted source images for qcow2 https://review.opendev.org/c/openstack/nova/+/919992
#openstack-nova - 2024-05-19
13:49:54 opendevreview Takashi Kajinami proposed openstack/osc-placement master: Replace simplejson by built-in json https://review.opendev.org/c/openstack/osc-placement/+/919999
14:34:57 opendevreview Dmitriy Rabotyagov proposed openstack/nova master: Handle MAC dash-addresses during migration https://review.opendev.org/c/openstack/nova/+/919760
#openstack-nova - 2024-05-20
11:56:10 wncslln hey nova o/, I was looking at some low hanging fruits bugs to solve, I've found this one https://bugs.launchpad.net/nova/+bug/2065693
11:56:32 wncslln since I'm a beginner, someone could show me where I should fix, if is possible?
11:57:35 wncslln I undestood that foreign key may does not exist in another table
12:04:14 sean-k-mooney wncslln: that might not be the best bug to start with
12:05:17 sean-k-mooney this tempest test is trying to delete an server that has not been completed spawning yet
12:06:01 sean-k-mooney what failing is recordign the delete event on the instancee because the instnace does not currently exist in the instnace table
12:06:23 sean-k-mooney that is causing the foreign key to fail
12:06:47 sean-k-mooney so this is not an issue with the db schema
12:07:44 sean-k-mooney whats broken here is that we shoudl gracefuly handle that issue and either reject the delete at the api or complete the delete without an error
12:10:56 sean-k-mooney neither are trivial to do correctly. we likely do no want to reject the request as a user should always be able to delete an insance for billing reasons
12:11:19 sean-k-mooney so while a 409 conflict could be valid in other case it woudl be undesireable in this case
12:11:58 sean-k-mooney the simplest possibel fix in this case would be ot catch the excption to record the instance action and log/ignore it then proceeed with the delete
12:13:29 sean-k-mooney that would be a valid resolution to the bug but if the instance does not exist in the cell db it also means we cant mark it as deleted which means we have no way to interup the request to spawn it.
12:13:42 sean-k-mooney which is why this is a hard thing to fix as a first bug
12:15:29 wncslln I see... I'll follow the bug updates to see the resolution and try to understand it
12:16:29 wncslln so sean-k-mooney, do you recommend any bug?
12:17:06 sean-k-mooney we do actully have a list of low hanging fruit bugs https://bugs.launchpad.net/nova/+bugs?field.tag=low-hanging-fruit i have not looked at that in a while
12:17:11 sean-k-mooney but perhaps one of those
12:18:31 sean-k-mooney perhaps https://bugs.launchpad.net/nova/+bug/2015330
12:19:40 sean-k-mooney for something perhaps more featury then bug we have this tech debt reduction bug https://bugs.launchpad.net/nova/+bug/1994139
12:22:57 wncslln I'll take a look on these. thanks!
13:36:36 stan live migrate says rolled back on the target server, what would be the appropriate logs to check for further troubleshooting?
14:36:29 opendevreview sean mooney proposed openstack/nova master: [WIP] ensure correct cleanup of multi-attach volumes https://review.opendev.org/c/openstack/nova/+/916322
14:52:12 dansmith bauzas: trivial patches to CI jobs, if you have a minute: https://review.opendev.org/c/openstack/nova/+/919738
14:52:33 dansmith first one saves us *minutes* of setup time on the affected jobs, second one goes back to regular images as we discussed
15:24:22 opendevreview Dan Smith proposed openstack/nova master: Enable OCaaS for several nova jobs https://review.opendev.org/c/openstack/nova/+/919738
15:24:22 opendevreview Dan Smith proposed openstack/nova master: Stop using split UEC image (mostly) https://review.opendev.org/c/openstack/nova/+/919739
15:24:28 dansmith melwitt: ^
15:24:35 dansmith good catch!
15:32:58 dansmith melwitt: thanks!
15:33:38 melwitt np

Earlier   Later