Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-03-16
10:33:54 songwenping Rocky and PciPassthroughFilter: (start: 1, end: 0)
10:34:52 sean-k-mooney that should work
10:34:55 bauzas you would need DEBUG levels
10:35:09 sean-k-mooney songwenping: can you check the pci_devices table in the nova cell db
10:35:14 sean-k-mooney and make sur eboth are there
10:35:30 sean-k-mooney songwenping: you alos need to make sure xorg (if its installed) is not using one of the gpus
10:35:41 sean-k-mooney i.e. make sure they are both bound to vfio_pci
10:35:57 songwenping pci_devices table has two gpu record.
10:36:04 sean-k-mooney ok
10:36:20 sean-k-mooney and one is aviable
10:36:36 sean-k-mooney and the other asigned to the first vm
10:36:43 bauzas sean-k-mooney: that's a scheduler filter issue
10:36:51 bauzas so not related to the driver
10:36:53 sean-k-mooney ya it is but i dont think its a bug
10:37:13 bauzas yup
10:37:15 sean-k-mooney bauzas: right i jsut mentioned the driver for after this is fixed
10:37:30 bauzas anyway, I need to get off the keyboard
10:37:33 songwenping i am opening the debug level to get more info
10:37:35 sean-k-mooney songwenping: are you useing numa by the way
10:37:48 sean-k-mooney i.e. cpu pinning or hugepages
10:37:49 songwenping no using numa
10:37:54 sean-k-mooney ok
10:38:36 sean-k-mooney if you can pastebin the entires for the pci_devices and debug secduler logs we can take a look
10:39:09 sean-k-mooney i just want to look at the 2 for the gpu
10:39:38 sean-k-mooney its odd that one worked and the other is failing
10:39:47 sean-k-mooney i would have expect both to fail or both to work
10:40:23 sean-k-mooney im wonderign what device_type is set to i assum type-PCI but it could be type-PF depending on the type of gpu
10:55:25 songwenping sean-k-mooney,bauzas: that's becuase the disk is not enough, i use the local disk. thanks a lot.
10:56:21 sean-k-mooney huh the pci filter shoudl not be
10:56:36 sean-k-mooney oh the disk filter elimiated the host that had the free pci device
10:56:49 sean-k-mooney and then the pci filter elimiated the remaining host because there was not a free gpu there
10:56:53 sean-k-mooney ok makes sense
10:57:13 songwenping yes, the filter error is mislead.
10:57:38 sean-k-mooney ya you bacially need to look at the set of all filters when debugging it
10:57:48 sean-k-mooney not just the last one that elimiated the final hosts
10:58:28 songwenping yeah the debug info is very useful.
10:58:55 sean-k-mooney some operators run with dbug on all the time for what its worth
10:59:05 sean-k-mooney you might now want it enabeld on all the services
10:59:14 sean-k-mooney but i like having it on for the scheduler at least
10:59:31 bauzas ok gtk
10:59:52 sean-k-mooney there is an argument to be made for moving those specific logs to info level
11:00:15 sean-k-mooney we didnt in the past because it can be a bit noisy but its a trade off
11:11:11 Uggla sean-k-mooney, gibi, bauzas Hi, is the a way to disable a flag set with self.flags(...) ?
11:15:41 sean-k-mooney Uggla: well you can just set it to what you want but if you mean you want to revert it to the unset state then you likely need to do thave via oslo
11:16:20 sean-k-mooney https://docs.openstack.org/oslo.config/latest/reference/locations.html
11:16:33 sean-k-mooney so flags i think is calling set_override
11:17:05 sean-k-mooney https://docs.openstack.org/oslo.config/latest/reference/api/oslo_config.html#oslo_config.cfg.ConfigOpts.reset
11:17:09 sean-k-mooney there is a reset funciton
11:17:25 sean-k-mooney so you would do CONF.my_section.my_var.reset()
11:17:38 Uggla sean-k-mooney, cool I'm going to have a look. Thanks
11:17:39 sean-k-mooney that will clear any orverdie ro default value
11:18:00 sean-k-mooney reset()¶
11:18:00 sean-k-mooney reset()¶
11:18:02 sean-k-mooney Clear the object state and unset overrides and defaults.
11:19:06 sean-k-mooney Uggla: feel free to wrap that up in a reset_flags helper funciton if you want to use it
11:20:02 sean-k-mooney i have never needed to od this in a test before so before you do are you sure you need this
11:20:38 sean-k-mooney usually i woudl sugges makeign sure the parent of your test class does not do thing you dont need
11:21:02 sean-k-mooney instead of trying to reset teh config state
11:23:59 Uggla sean-k-mooney, I'm not sure I try to have a src node configure with a flag and a dest node without. But it seams that flags are 'global' in functional test.
11:24:13 sean-k-mooney oh you cant do that
11:24:25 sean-k-mooney its a limiation of our test infra
11:24:47 sean-k-mooney there is a way to make it work with context manger which i demonstarted a few years ago but we decided not to do that
11:25:31 sean-k-mooney if your doing this in a functional test you will need to mock it differntly
11:25:54 Uggla sean-k-mooney, the idea is to simulate a live migration from src node with a flag to dst node without this flag
11:26:17 sean-k-mooney yes you cant do that today in our func tests without a lot of hacks
11:29:27 Uggla sean-k-mooney, yes I manage to do what I wanted with mock. But I was wondering if something simpler can be used.
11:29:47 Uggla Is the reset proposed above a potential way to do that ?
11:30:02 sean-k-mooney not really
11:30:10 sean-k-mooney when you reset it it will reset it for both computes
11:31:58 Uggla sean-k-mooney, in my case I think it will be ok, I'll try
11:32:16 sean-k-mooney it likely will be racy
11:32:35 sean-k-mooney basicaly this is something we currently try not to do
11:32:52 sean-k-mooney so if you get ti workign we will have to review it very carefully
11:38:43 sean-k-mooney Uggla: https://stackoverflow.com/questions/33504543/passing-import-context-variables-to-decorators/33507308#33507308
11:42:40 sean-k-mooney thats not quite what you want but i had an example for this in nova somewhere in gerrit with context manager but i cant find it
13:02:44 opendevreview Alexey Stupnikov proposed openstack/nova stable/victoria: Cleanup old resize instances dir before resize https://review.opendev.org/c/openstack/nova/+/864730
13:33:42 opendevreview Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648
13:33:42 opendevreview Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446
13:37:01 opendevreview Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446
17:32:15 opendevreview Amit Uniyal proposed openstack/nova master: Allow swap resize from non-zero to zero https://review.opendev.org/c/openstack/nova/+/857339
18:12:21 darkhorse hi team, i noticed that the periodic tasks defined with @periodic_task decorator at scheduler/manager.py performs tasks on all workers instead of running once at defined spacing. is that a bug or a designed behavior?
18:12:30 darkhorse hi team, i noticed that the periodic tasks defined with @periodic_task decorator at scheduler/manager.py performs tasks on all workers instead of running once at defined spacing. is that a bug or a designed behavior?
18:24:13 dansmith darkhorse: which periodic are you looking at? but yes, it's *expected* and the periodic should use some locking to prevent it from running simultaneously if need-be
18:26:45 darkhorse dansmith: thank you for clarification. i have added a custom periodic task that should run every minute. will you point me to an example code that uses locking?
18:27:05 dansmith darkhorse: oslo lockutils :)
18:27:26 dansmith with external=true so it uses a lock file, which all the others will see
18:27:38 dansmith if not external, then it'd just be per-process, which wouldn't help
18:29:43 darkhorse ok thank you!
#openstack-nova - 2023-03-17
09:32:35 opendevreview René Ribaud proposed openstack/nova master: Reproducers for bug 1869804 https://review.opendev.org/c/openstack/nova/+/877772
09:32:36 opendevreview René Ribaud proposed openstack/nova master: Fix for bug 1869804. https://review.opendev.org/c/openstack/nova/+/877773
16:23:41 opendevreview René Ribaud proposed openstack/nova master: Reproducers for bug 1869804 https://review.opendev.org/c/openstack/nova/+/877772
16:23:41 opendevreview René Ribaud proposed openstack/nova master: Fix for bug 1869804. https://review.opendev.org/c/openstack/nova/+/877773
#openstack-nova - 2023-03-18
05:04:23 opendevreview Merged openstack/nova stable/wallaby: Improving logging at '_allocate_mdevs'. https://review.opendev.org/c/openstack/nova/+/871416
07:26:59 opendevreview Merged openstack/nova stable/victoria: Retry attachment delete API call for 504 Gateway Timeout https://review.opendev.org/c/openstack/nova/+/866087
08:37:49 opendevreview Vasyl Saienko proposed openstack/placement master: Do not use coalesce for consumers.uuid https://review.opendev.org/c/openstack/placement/+/877852
#openstack-nova - 2023-03-20
10:09:39 dvo-plv sean-k-mooney: Hello, Are you here ?
10:31:15 opendevreview Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446
12:01:56 sean-k-mooney stephenfin: speaking of the sdk i proposed this revert https://review.opendev.org/c/openstack/python-openstackclient/+/877017 mainly for getting some discussion at the ptg
12:02:27 stephenfin sean-k-mooney: I thought I replied to that. Evidently not.
12:03:56 stephenfin I raised the concerns when it merged but we decided to get it in while we still had students working on it. That's relatively easy to fix without removing SDK support, thankfully. Just needs someone to do it

Earlier   Later