Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-09-03
16:07:50 bauzas gibi: thanks for the summary
16:07:54 sean-k-mooney that could cause import issues like that
16:08:31 sean-k-mooney gibi: ah sorry i ment to join i got nerd sniped by somethign else
16:09:47 sean-k-mooney gibi: by the way i almost never use our fake driver, but if we wanted to test a mroe compelte fake driver eventually we coudl likely enabel the libvirt fixture in it vai a config option or siilar. it still would not actully talk to libvirt but it would be much more complete
16:10:04 sean-k-mooney basiclly run it the way we do in the libvirt fucntional tests
16:10:35 sean-k-mooney i dont think you can do that via config today but it could be intersting to try
16:11:09 sean-k-mooney kolla has or at least had the ablity to deploy multiple copies of the nova and neutorn fake drivers per physical host in the past
16:11:39 sean-k-mooney allowing you to take 3 host and run 10s to 100s of fake hosts
16:12:22 sean-k-mooney i think devstack might eb able to do that too but its been 8 years or so since i actully tried to use any of that
16:14:16 gibi yeah it is a good idea to try to upgrade our fake driver to use the libvirt fixture. I have an item for G to have fake driver based scale testing that can be extended with this
16:16:25 sean-k-mooney https://docs.openstack.org/devstack/latest/guides/nova.html#fake-virt-driver
16:17:27 sean-k-mooney so we do still have the basic supprot but i dotn knwo if this ever worked with ovn adn i suspect there coudl be other issues.
16:17:41 sean-k-mooney just bit rot honstly more then anything fundemental
16:18:21 sean-k-mooney but the ml2 ovsagent can run in a fake mode, the other hack is to just test without networking
16:20:02 opendevreview ribaudr proposed openstack/nova master: Add Flamingo prelude section https://review.opendev.org/c/openstack/nova/+/959188
16:22:17 gmaan Uggla: you need quick fix here which is caused by rebase, btw updates lgtm now - https://review.opendev.org/c/openstack/nova/+/952894/comment/75fbb034_6c75825e/
16:24:44 Uggla gmaan, thx I did not noticed it failed. I gonna fix it right now.
16:48:48 opendevreview ribaudr proposed openstack/nova master: Reproducer for bug 2114951 https://review.opendev.org/c/openstack/nova/+/952894
16:48:49 opendevreview ribaudr proposed openstack/nova master: Fix bug 2114951 https://review.opendev.org/c/openstack/nova/+/952895
16:59:23 opendevreview Merged openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994
17:29:26 opendevreview Merged openstack/nova master: Fix 'nova-manage image_property set' command https://review.opendev.org/c/openstack/nova/+/946733
18:38:07 gmaan Uggla: thanks +2 on both
19:55:04 zigo sean-k-mooney: It's normal to do PYTHONPATH=., because the package isn't installed in /usr/lib/python3/dist-packages when calling sphinx, that's a way to add the module in the path.
19:55:14 zigo I do that very often in my packages, with no issue.
19:55:43 zigo As for my error here, the weird thing is that it happens it crashes in Unstable, but not in Trixie, so something must have changed.
19:55:54 zigo I'll try to do a pip freeze and compare.
19:58:28 clarkb zigo: why not add the path to the location rather than .?
19:59:12 clarkb then cmd would only be resolvable as tempest.cmd or the stdlib stuff
20:31:47 zigo clarkb: It produces the same result, that's not the issue.
20:33:14 clarkb zigo: I think that means you're a directory level too deep?
20:33:45 clarkb youre in tempest/ so cmd is there and . allows you to import cmd that way. Another approach would be to put it at the end of your path if python resolves the path in order
20:33:53 zigo I'm not.
20:33:58 zigo I'm on top of that folder.
20:39:07 clarkb its possible that the sphinx build changes its cwd maybe? (but I would expect the rooted version instead of '.' to fix that)
20:40:50 zigo Ok, the diff is because of sphinx version.
20:41:19 zigo 8.1.3 in Trixie, vs 8.2.3 in Unstable.
20:41:58 zigo I checked building in Trixie, adding sphinx as --extra-package in the sbuild command line.
20:42:17 zigo And then it fails ...
20:42:38 zigo clarkb: Is there a way I can do such demonstration with a patch in gerrit?
20:46:03 zigo In conf.py, there's:
20:46:03 zigo sys.path.insert(0, os.path.abspath('../../tempest'))
20:46:26 zigo That's a likely guilty thingy ... :P
20:46:49 clarkb it could be just run the tempest doc job with the version of sphinx that you want
20:47:06 clarkb I think you might need to edit requirements upper-constraints and do a depends on for that
20:47:45 clarkb its possible that should be os.path.abspath(os.path.join(__file__, '../../'))
20:47:55 clarkb then you have to refer to things as tempest.cmd not just cmd
20:49:15 zigo BINGO !!!
20:49:17 zigo I was right.
20:51:25 zigo I see no reason why the conf.py does this ... When I remove it, building the doc continues to work.
20:51:59 clarkb the git log might give a hint. But if it works without it then removing it seems like a good option
20:52:23 zigo Yeah, let me git blame.
20:52:59 clarkb If extensions (or modules to document with autodoc) are in another directory, add these directories to sys.path here. If the directory is relative to the documentation root, use os.path.abspath to make it absolute, like shown here.
20:53:28 clarkb thats from the comment in conf.py. So its related to autodoc. It is possible that it is required to make autodoc work and then that breaks sphinx trying to use cmd
20:55:07 clarkb zigo: if you look at the automodule directives they all start with tempest
20:55:35 clarkb so my hunch here is that the path manipulation is unnecessary (because its off by a level anyway) and that the doc build must be relying on tox installing the package before building
20:55:48 clarkb from https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#ensuring-the-code-can-be-imported the code just need to be importable
20:56:39 clarkb if you need to keep it then I think you just need __file__ + '../../' then tempest is in that directory and the automodule directives should find it is importable from there. But also we can stop polluting the nova channel as this is a qa problem
20:57:34 zigo Let's see how it goes: https://review.opendev.org/c/openstack/tempest/+/959390 :)
20:58:43 zigo clarkb: Yeah, wrong chan, appologies.
21:08:59 zigo Comes from here: https://review.opendev.org/c/openstack/tempest/+/735619
#openstack-nova - 2025-09-04
06:53:46 opendevreview Rajesh Tailor proposed openstack/nova stable/2025.1: Fix 'nova-manage image_property set' command https://review.opendev.org/c/openstack/nova/+/958834
08:33:58 opendevreview Balazs Gibizer proposed openstack/nova master: Reproduce bug/2115905 https://review.opendev.org/c/openstack/nova/+/954336
08:33:59 opendevreview Balazs Gibizer proposed openstack/nova master: [PCI tracker]Remove non configured devs when freed https://review.opendev.org/c/openstack/nova/+/954613
08:33:59 opendevreview Balazs Gibizer proposed openstack/nova master: [pci]Keep used dev in Placement regardless of dev_spec https://review.opendev.org/c/openstack/nova/+/954149
08:34:59 gibi sean-k-mooney: ^^ I fixed the commit messages and pulled out two constants for the log message asserts
08:36:23 sean-k-mooney[m] cool ill review it after i grab a coffee and start my day
09:01:48 mmessah Hi all, `/v2.1/os-hypervisors/detail` requests take too long time after upgrade to Epoxy. I saw similar issue for server list like https://bugs.launchpad.net/nova/+bug/2106380 but we dont have issue for server list. We tested manually with curl request. if I use OpenStack-API-Version is 2.88 and above, we wait about 20 seconds. if I use OpenStack-API-Version is 2.87, result is returned immediately. Is there anyone
09:01:48 mmessah experiencing this situation?
09:06:15 jkulik 2.88 added the "uptime" field into the hypervisor details. we experienced slowness with that, too ... and thus added a config setting downstream to patch it out: https://github.com/sapcc/nova/commit/686f44c4c65eba6c851c852804e82a3e00309945
09:11:41 frickler jkulik: interesting, that looks like a different source of slowness to me, did you ever try to upstream your patch? it does seem pretty reasonable to me (caveat I'm no nova developer, just another deployer)
09:12:39 jkulik frickler: no, it stems from a time with less upstream-involvement. especially after an upgrade, downstream times are quite ... stressful :) I'd be happy to propose that, though
09:13:11 masahito we hit the same issue from the v2.88. calling the RPC to all hypervisors in serial causes slow os-hypervisor/detail API response.
09:15:04 masahito large scale deployment hits the same issue. we just wait few minutes for 10K HV situation, or use limit and maker to separate API.
09:19:53 frickler yes, iterating over all hypervisors for an API call (assuming there's no caching, didn't check the code) sounds pretty much not scalable
09:26:49 jkulik hm ... Isn't there some policy that the API response shouldn't change depending on operator configs? our downstream patch would break that policy.
09:28:16 jkulik api-ref says "Only reported for active hosts where the virt driver supports this feature." so ... maybe it's still fine, because clients would have to handle those cases anyways?
09:34:18 bauzas hey folks, do we have any upstream tracker for that bug ?
09:34:41 bauzas just overlooked the code and yeah I'm afraid of the fact this is indeed a performance issue
09:35:10 bauzas when calling _view_hypervisor(), we then call the host API for each of the hypervisors
09:35:16 bauzas gibi: ^
09:35:21 jkulik I haven't found one. Also no review-request.
09:36:18 bauzas https://github.com/openstack/nova/blob/9f156aa954fb57eeaf407cc13a0f782b8af1a5d3/nova/api/openstack/compute/hypervisors.py#L100-L101
09:40:47 opendevreview Johannes Kulik proposed openstack/nova master: Conditionally disable hypervisor uptime in details API https://review.opendev.org/c/openstack/nova/+/959420
09:42:15 jkulik ready to work on making ^ better. I guess it needs a releasenote at least? but before that, I'd like to get some feedback if this approach should be continue
09:42:51 bauzas eek, https://github.com/openstack/nova/blob/9f156aa954fb57eeaf407cc13a0f782b8af1a5d3/nova/virt/libvirt/driver.py#L12319-L12322
09:43:11 bauzas so now when calling statistics, we do N calls to an external process call
09:43:39 bauzas jkulik: I need to review your patch but we can't have API behaviour changed by configuration
09:44:13 jkulik yeah, figured. it's already behaving like that for a subset of the HVs, though
09:45:15 bauzas hmmm, unfortunately I need to leave by the next mins
09:46:07 bauzas jkulik: where do we conditionally change the API behaviour ?
09:46:12 bauzas I mean, already
09:46:33 jkulik depending on if the calls succeeed or rais NotImplemented, we return None instead of an actual uptime string
09:46:50 jkulik so this patch changes to "always None"
09:48:00 bauzas atm, if we return None, this is not because we change the configuration, right? this is just whether the call works or not
09:48:28 bauzas the concern with an confiugrable API is about interop
09:48:29 jkulik yes. for some hypervisors, this call never works. api-ref states for "uptime": "Only reported for active hosts where the virt driver supports this feature."
09:49:03 jkulik so imho, a client cannot rely on this being set
09:57:03 stephenfin jkulik: You're focused on the vmware driver, right? And I assume the noise you're talking about is the tracebacks in nova-compute? Couldn't you just implement the method for that driver but return None?
09:57:20 stephenfin That could likely be backported too
10:01:40 jkulik sure. doesn't bring any speedup at all though, so the _actual_ issue of os-hypervisor/detail being slow doesn't go away

Earlier   Later