Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2026-07-30
16:40:21 sean-k-mooney im runing the previous version instead to see if it fails locally
16:40:35 jgilaber oh I used the module instead of the function
16:40:48 jgilaber I'll fix it and move it to the base TestCase
16:44:07 opendevreview Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299
16:44:08 opendevreview Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840
16:47:31 gibi sean-k-mooney: melwitt: so yeah the intention is to have the file backed SQL only for the threading version of the jobs
16:48:19 gibi but in threading mode we do have a global lock today
16:48:36 sean-k-mooney is that global lock hsared btween nova an placmeent?
16:48:44 sean-k-mooney or one per service
16:48:45 gibi and we have a patch to change that to a per DB lock https://review.opendev.org/c/openstack/nova/+/992862
16:48:57 gibi yeas it is a shared global lock I think
16:49:02 sean-k-mooney ack
16:49:11 gibi but only should be in place for the threading tests
16:49:13 sean-k-mooney so that could expalin some of the warnings and slowness
16:49:16 gibi threading version of the test jobs
16:49:24 sean-k-mooney oh right
16:49:31 sean-k-mooney this was in the eventlet based job
16:49:31 gibi we can land https://review.opendev.org/c/openstack/nova/+/992862 and see if it helps
16:50:17 sean-k-mooney i mena i can go review that now in either case
16:50:28 gibi we could add some tracing to the lock handling and see if that shows that we somehow introduced the lock to the eventlet job as well
16:50:30 sean-k-mooney runing jgilaber's patch locally its not repoduceing for me
16:51:35 sean-k-mooney hum
16:52:03 sean-k-mooney gibi: so this woudl be a good case to use the FiarLockGuarde i wrote
16:52:49 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/utils.py#L1149
16:59:48 gibi I'm not following how
17:00:12 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/992862/comment/6022582b_82867746/
17:01:06 gibi but fairlockguard is to handle when multiple locks is being taken in an order
17:01:13 gibi here we only take one lock
17:01:16 sean-k-mooney no
17:01:22 sean-k-mooney that a usecase
17:01:38 sean-k-mooney it was intened to be used anytime you need to aquire the same named lock in diffent context
17:01:46 sean-k-mooney not just multipel locks
17:03:42 gibi I feel like here we have a simpler use case than what the fairlockguard is created for. Sure fairlockguard is also capable of doing this but to do this we don't necessary need the complexity of the fairlockguard
17:04:04 sean-k-mooney right but i wnated ot creat that so we would stop using one of version of it
17:04:24 sean-k-mooney this is reimpelnted a new lock guard
17:05:36 sean-k-mooney we dont have to use it i think it woudl be good to use it when we need soemthing like this in the futre
17:06:40 gibi it was a long day so I might not capable of seeing the whole picture any more but fairlockguard could be used in any place where a single lock is taken, but we don't want to replace all the locks in our code with a fairlockguard as that would be overkill
17:07:20 gibi fairlockguard shines where you have a list of lock to be taken
17:07:26 gibi and you want to do that safely
17:07:50 gibi if you have just a single lock to take at a time then it is overkill in my eyes
17:08:23 sean-k-mooney the bit i dont like is the fact your creating collections.defaultdict(threading.RLock)
17:08:31 sean-k-mooney and locking that collection access
17:08:48 sean-k-mooney that changed it form a singel lock to a set of related locks
17:09:40 sean-k-mooney even if any one invoction of the _locked_scope context manager is only useing one lock
17:12:40 gibi we could monkey patch the lock object into transaction_context_manager._root for each DB then the DatabaseWriteLock does not need to look up the lock in a dict that maps the context manager to a lock
17:13:15 sean-k-mooney we could im not aginst merging the patch as it is
17:13:24 gibi but as we already hooking into sqla here it would be extra complexity to hook this deep into it again at DB creation
17:13:47 sean-k-mooney but we spent a lot of effort addign nesting suprot and other infra in the fairlockgurard and testing to verify it for complex lockign cases like this
17:15:31 gibi but exactly those things are not needed in this case
17:15:37 gibi (I think)
17:15:59 gibi it is not such a complex case
17:16:00 sean-k-mooney you have muyltiple related locks, nesting / rentrancy
17:16:11 sean-k-mooney hence the callout why you needed to RLOCKS
17:16:16 gibi these locks are not related each is for its own DB
17:16:59 gibi they are not guarding the same shared resouce it is one lock - one DB
17:17:07 sean-k-mooney i know
17:17:28 sean-k-mooney but you all out thing like "An RLock is used per database because a single thread may nest writer
17:17:30 sean-k-mooney transactions against the same engine facade."
17:18:20 gibi and that is exactly what an RLock is for :)
17:18:48 sean-k-mooney and its also one of the thing build into the FairlockGurads constaitns
17:18:54 sean-k-mooney ill upgarde to a +2 for now
17:19:19 sean-k-mooney but if this proves unstabel i would prefrer ot adotp the FairLockGuard instead fo theying ot fix this implemetion
17:19:28 sean-k-mooney gibi: does that work for you
17:20:54 gibi if the instability is due to a problem that is solved by the fairlockguard then yes it is good for me
17:22:28 sean-k-mooney patchset 8 was created 21 days ago
17:22:43 sean-k-mooney so that before your shutdown speedup patch landed yes
17:23:11 sean-k-mooney so we woudl execte a new run to also be faster
17:23:47 sean-k-mooney or rather the thereaded and eventet version shoudl be closer now
17:41:23 jgilaber sean-k-mooney, so after clearing the cache the functional tests are green again https://zuul.opendev.org/t/openstack/buildset/6a7d3edc4ae84ba7b33487e037f1acc5
17:41:50 sean-k-mooney jgilaber: with an without that change i was not able to cause it to fial locally
17:41:56 sean-k-mooney i still have it runingin the backgorund
17:42:20 sean-k-mooney but if ci is happy i guess that is progress
17:43:31 jgilaber yeah, I don't like that I don't know what was wrong there, but good to advance
17:44:15 jgilaber I guess CI harware is slower than our local machines and caused the problem to surface somehow
17:44:58 sean-k-mooney yes that happens
17:45:44 sean-k-mooney you can sort of emulatie that by messign with some of the test teimouts
17:46:03 sean-k-mooney but then its hard to tell if tis a real failure or not
17:47:59 gmaan gibi: you have time to talk about unify the executors get/destroy comment https://review.opendev.org/c/openstack/nova/+/998571/comment/23f68a57_21843d10/
17:48:16 gmaan if you are done for the day, I can add comment on gerrit and we can discuss there or tomorrow
18:01:01 opendevreview Merged openstack/nova master: tests: Use per-database write locks instead of global lock https://review.opendev.org/c/openstack/nova/+/992862
19:22:58 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Fix deny_share host locality for NFS https://review.opendev.org/c/openstack/nova/+/998752
19:22:58 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Support cold migration with virtiofs shares https://review.opendev.org/c/openstack/nova/+/989633
19:54:06 Zhan[m] Hi friends, recently I found a somewhat tricky bug with live migration: https://bugs.launchpad.net/nova/+bug/2160728. Wondering if I can get some feedbacks on it and I can work on the fix too, thanks!
22:47:33 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048
#openstack-nova - 2026-07-31
01:45:38 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Support cold migration with virtiofs shares https://review.opendev.org/c/openstack/nova/+/989633
01:45:39 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Extract share management from ComputeManager https://review.opendev.org/c/openstack/nova/+/999349
02:02:22 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Extract share management from ComputeManager https://review.opendev.org/c/openstack/nova/+/999349
02:02:23 opendevreview Goutham Pacha Ravi proposed openstack/nova master: Support cold migration with virtiofs shares https://review.opendev.org/c/openstack/nova/+/989633
07:35:17 gibi gmaan: sorry I was already done for the day at that point yesterday.
08:03:40 opendevreview Roman Hros proposed openstack/nova master: Delete inactive port bindings during live migration https://review.opendev.org/c/openstack/nova/+/999374
08:25:16 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: check tempest for instability https://review.opendev.org/c/openstack/nova/+/999378
09:30:53 opendevreview Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299
09:30:53 opendevreview Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840
09:34:32 opendevreview Roman Hros proposed openstack/nova master: Delete inactive port bindings during live migration https://review.opendev.org/c/openstack/nova/+/999374
10:20:32 opendevreview Roman Hros proposed openstack/nova master: Delete inactive port bindings during live migration https://review.opendev.org/c/openstack/nova/+/999374
10:34:15 ralonsoh sean-k-mooney, hi! will you have some time to talk about https://bugs.launchpad.net/nova/+bug/2161894?
10:34:38 ralonsoh In particular about `create_tap`!=`ovs_create_tap`
10:38:21 sean-k-mooney sure am are you free now ish, or at the top of the hour?
10:38:34 sean-k-mooney i was going to go grab a coffee quickly
10:38:56 sean-k-mooney but we can chat in say 5 mins

Earlier   Later