| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-06-20 | |||
| 17:15:21 | JayF | if nova is seeing something similar, I may file an upstream eventlet issue | |
| 17:31:30 | frickler | JayF: well the log I linked to above was from bauzas testing nova on Noble, so running with py3.12 too. I'd need to double check that it doesn't happen on py3.11 | |
| 17:34:03 | frickler | hmm, opensearch does find some hits in toy-py311 jobs, too, like https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ce6/922421/1/check/openstack-tox-py311/ce64a27/job-output.txt | |
| 17:34:18 | JayF | they almost always appear in unit tests | |
| 17:34:22 | sean-k-mooney | frickler: i was able to stack on noble fine yestereday | |
| 17:34:22 | JayF | they are only relevant for real runs | |
| 17:34:28 | sean-k-mooney | i have that runnign in a vm currently | |
| 17:34:36 | sean-k-mooney | but i didn try ot use the deploy openstack | |
| 17:38:29 | sean-k-mooney | frickler: i can boot a vm and do a console log show | |
| 17:39:21 | frickler | JayF: ah, o.k., the number found in those unit tests seems to go from 1 to 4 though from 3.11 to 3.12. also not sure neutron-functional job falls https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_cec/917430/8/check/neutron-fullstack-with-uwsgi/cec5565/controller/logs/services.txt | |
| 17:39:37 | frickler | other than that the only hit is the new noble job | |
| 17:41:04 | sean-k-mooney | im sure there are proably bugs but at first glance things appear to work fin on noble and python 3.12 | |
| 17:41:04 | frickler | sean-k-mooney: yes, what triggered me to look at the logs were the tempest failures in the mtty job. but they seem to be related to the actual mtty stuff afaict | |
| 17:41:26 | JayF | My current working hypothesis is it's a reporting issue, not a functional one. | |
| 17:41:46 | sean-k-mooney | frickler: ah ok well i htink there is a bug in sylvains nova patch | |
| 17:41:54 | sean-k-mooney | or somehting related | |
| 17:42:19 | frickler | JayF: yes, at least I have no data that would say otherwise, but would be great if we could be sure of that | |
| 17:42:22 | sean-k-mooney | the tempet test are failign because nova is not reporting an resouce provider inventireos for the mtty devices | |
| 17:42:57 | sean-k-mooney | frickler: the only thing i had to do was pip install -U setuptools in the requiremend and global venv | |
| 17:43:14 | sean-k-mooney | i started on a patch to do that in devstack yesterday but didnt get back to it | |
| 17:43:41 | sean-k-mooney | has anyoen proposed a noble job to the devstack repo yet? | |
| 17:43:58 | sean-k-mooney | if not i can maybe do that this week | |
| 17:45:17 | frickler | ah, yes, I still wanted to add some unit test for edit-constraints to the reqs fix | |
| 17:49:25 | opendevreview | sean mooney proposed openstack/nova master: [WIP] split monkey_patching form import https://review.opendev.org/c/openstack/nova/+/922425 | |
| 18:08:54 | sean-k-mooney | frickler: JayF https://review.opendev.org/c/openstack/devstack/+/922427 should work? | |
| 18:09:21 | JayF | nice, I can hook that up to ironic once it shows it's working elsewhere | |
| 18:09:25 | JayF | would be interesting to see what's busted | |
| 18:09:44 | sean-k-mooney | maybe | |
| 18:09:59 | sean-k-mooney | i just got back to the eventlet stuff after 4 weeks or so | |
| 18:10:15 | sean-k-mooney | so right now im trying to load back up the context on what i was doing | |
| 18:12:47 | sean-k-mooney | JayF: ill generally be moving most of my dev envs over to noble or c10s over the next few weeks, both will be using python 3.12 | |
| 18:13:13 | JayF | Honestly, it's just a fluke I caught this. 3.12 is my default python, I was testing a local dev thing for Ironic, and just saw it | |
| 18:13:48 | sean-k-mooney | i feel like that how we catch most things | |
| 18:14:09 | sean-k-mooney | one of use runs a ill advicsed setup (i.e. the latest and gratest of whatever) and fine odd bugs | |
| 18:15:12 | sean-k-mooney | in anycase that should give use a full tempest-full run on noble without any other chagnes if it stacks properly | |
| 18:15:27 | frickler | sean-k-mooney: while just testing you may also want to drop all the other jobs in the check pipeline | |
| 18:15:29 | JayF | I was about to make some snarky comment about being offended you find my gentoo laptop ill advised ... then I realized you're *probably* actually right :D | |
| 18:15:31 | sean-k-mooney | whihc shoudl be antoher datapoint to if this is an actul problem or jsut a logging issue | |
| 18:15:47 | sean-k-mooney | frickler: yep i remmebered that after i pushed | |
| 18:15:55 | sean-k-mooney | frickler: ill do it if i respin | |
| 18:16:04 | frickler | ack | |
| 18:19:11 | sean-k-mooney | pbr still depends on distutils by the way | |
| 18:19:12 | sean-k-mooney | https://opendev.org/openstack/pbr/src/branch/master/pbr/core.py#L51 | |
| 18:20:34 | sean-k-mooney | so until we drop that or we start adding seutuptools > whatever | |
| 18:20:49 | sean-k-mooney | to the relevent project that depend on distutils | |
| 18:20:57 | sean-k-mooney | we will need to keep the workarouncx in devstack | |
| 18:33:20 | opendevreview | Rafael Weingartner proposed openstack/nova master: Block VM instance resize to flavor with different aggregates https://review.opendev.org/c/openstack/nova/+/922421 | |
| 18:35:37 | opendevreview | Rafael Weingartner proposed openstack/nova master: Block VM instance resize to flavor with different aggregates https://review.opendev.org/c/openstack/nova/+/922421 | |
| 18:38:45 | clarkb | sean-k-mooney: PBR also depends on setuptools (which caries distutils these days) so you're depending on both really | |
| 18:39:11 | sean-k-mooney | clarkb: i didnt see the requrieemtns file for pbr in the repo | |
| 18:39:20 | sean-k-mooney | did i miss it or is that depenciy specified elsewher | |
| 18:39:47 | clarkb | sean-k-mooney: its implicit as pbr is a setuptools setup_requires hook or whatever its called | |
| 18:39:50 | sean-k-mooney | there are no deps in https://github.com/openstack/pbr/blob/d03d617c09e7ba8ddf62d1e53d71685cd708e2da/setup.cfg or https://github.com/openstack/pbr/blob/d03d617c09e7ba8ddf62d1e53d71685cd708e2da/setup.py | |
| 18:40:04 | clarkb | you can't list it as a requirment because of chicke and egg issues. The pyproject.toml documentation does explicitly list setuptools though | |
| 18:40:15 | sean-k-mooney | clarkb: ok but without an explict depency we can specify a min version | |
| 18:40:36 | sean-k-mooney | right here https://github.com/openstack/pbr/blob/d03d617c09e7ba8ddf62d1e53d71685cd708e2da/pyproject.toml.future#L7 | |
| 18:40:44 | clarkb | pbr is expected to run with a large range of setuptools versions | |
| 18:40:54 | clarkb | there probably is some min version but its going to be old | |
| 18:41:29 | sean-k-mooney | we basically need to say if python 3.12 reuqire setuptools >= the one with distutils | |
| 18:41:43 | sean-k-mooney | which we would normaly do in a requriemtes.txt | |
| 18:41:53 | sean-k-mooney | with the version spericifc marker | |
| 18:42:31 | clarkb | right but you can't because of the chicken and egg problem. The only solution to that is to use pyproject.toml instead | |
| 18:42:39 | clarkb | but no projects are doing that yet as far as I know | |
| 18:42:39 | sean-k-mooney | i.e. setuptools>=70;python_version='3.12' | |
| 18:43:09 | sean-k-mooney | clarkb: well i started adding pyproject.toml support | |
| 18:43:25 | sean-k-mooney | but it need the wsgi_srcipts to be replaced | |
| 18:43:32 | sean-k-mooney | so that work kind of stalled | |
| 18:43:36 | clarkb | basically the dependency may exist but you can't encode that in a requirements file in a way that works/makes sense. You can encode it in project.toml build requires though | |
| 18:44:27 | sean-k-mooney | well for now we can just have devstack install it | |
| 18:44:56 | sean-k-mooney | for porject other then pbr if they have a dep on distutils they can sepciry the requireemnt in the normal way | |
| 18:45:31 | clarkb | yes, the workaround there is appropriate. I just wanted to call out that it isn't just for distutils. You also need setuptools proper which isn't included by default in modern python either | |
| 18:45:55 | sean-k-mooney | ack ya thats also true | |
| 18:47:29 | sean-k-mooney | looking at codesearch/hound there are more project then i would like importing distutils | |
| 18:48:02 | sean-k-mooney | .... inlcuing oslo.versionedobjects | |
| 18:48:38 | sean-k-mooney | we proably should look to remove that dep and replace it with somethign else | |
| 18:49:25 | sean-k-mooney | mostly we seam to be suing it for version parsing | |
| 18:49:34 | sean-k-mooney | which we can replace with microversion_parse | |
| 18:51:20 | sean-k-mooney | frickler: the noble run is currently on tempet https://zuul.openstack.org/stream/0b61b3b596554e4093acd6e17f61b26c?logfile=console.log | |
| 18:51:37 | sean-k-mooney | so thats looking promising | |
| 18:51:43 | sean-k-mooney | im going to leave it there for today | |
| 22:48:13 | opendevreview | sean mooney proposed openstack/nova master: split monkey_patching form import https://review.opendev.org/c/openstack/nova/+/922425 | |
| #openstack-nova - 2024-06-21 | |||
| 07:55:36 | jopdorp_ | Hi | |
| 07:56:13 | jopdorp_ | I was wondering if anyone has experience with H100 HGX systems , or maybe DGX and GPU passthrough to KVM VMs in nova | |
| 07:56:28 | jopdorp_ | We're seeing an issue where the nvidia-smi command hangs | |
| 07:57:06 | jopdorp_ | we do isntall fabricmanager, and I've tried some different configs of it. | |
| 07:57:29 | jopdorp_ | We're trying to pass one gpu per vm, on a host that has 8xH100 SXM gpus | |
| 07:57:49 | jopdorp_ | nv_open_q takes 100% of a single cpu core when nvidia-smi is invoked and hangs | |
| 11:45:59 | sean-k-mooney | jopdorp_: even if our downstream customers ask that question we would have to direct them to nvida supprot | |
| 11:46:22 | sean-k-mooney | what nvidia-smi does internally is really a black box that only they understand | |
| 14:02:39 | opendevreview | sean mooney proposed openstack/nova master: refactor cmd dir for non eventlet console_scripts https://review.opendev.org/c/openstack/nova/+/904424 | |
| 16:47:15 | opendevreview | sean mooney proposed openstack/nova master: update nova-next to use ubuntu 24.04 https://review.opendev.org/c/openstack/nova/+/922492 | |
| 17:35:04 | opendevreview | sean mooney proposed openstack/nova master: Remove nova debugger funcitonality https://review.opendev.org/c/openstack/nova/+/922496 | |
| 17:35:05 | opendevreview | sean mooney proposed openstack/nova master: Add IO Thread Pool Executor https://review.opendev.org/c/openstack/nova/+/922497 | |
| 18:03:39 | opendevreview | sean mooney proposed openstack/nova master: remove eventlet.tpool https://review.opendev.org/c/openstack/nova/+/905287 | |
| #openstack-nova - 2024-06-22 | |||
| 03:10:45 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: Add config option to require secure SPICE. https://review.opendev.org/c/openstack/nova/+/922544 | |
| 03:10:45 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: allow direct SPICE connections to qemu https://review.opendev.org/c/openstack/nova/+/922545 | |
| 03:10:46 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: allow concurrent access to SPICE consoles. https://review.opendev.org/c/openstack/nova/+/922546 | |
| 03:10:46 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: Add guest devices to support SPICE USB. https://review.opendev.org/c/openstack/nova/+/922547 | |
| 03:10:47 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: Optionally enable SPICE debug logging. https://review.opendev.org/c/openstack/nova/+/922548 | |
| 03:10:47 | opendevreview | Michael Still proposed openstack/nova master: WIP: libvirt: Optionally support sound when using SPICE. https://review.opendev.org/c/openstack/nova/+/922549 | |