| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-08-27 | |||
| 18:54:31 | opendevreview | Merged openstack/nova master: Use format_inspector from oslo https://review.opendev.org/c/openstack/nova/+/925025 | |
| 18:55:00 | dansmith | hoo bioy | |
| 19:06:20 | sean-k-mooney | ah cool so the tempest test change also merged | |
| #openstack-nova - 2024-08-28 | |||
| 01:07:15 | opendevreview | melanie witt proposed openstack/nova master: nova-manage: Add --max_rows option to 'db purge' command https://review.opendev.org/c/openstack/nova/+/927339 | |
| 07:50:30 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra specs for USB redirection. https://review.opendev.org/c/openstack/nova/+/927354 | |
| 07:50:51 | mikal | sean-k-mooney: that's the extra specs for USB device redirection finished I hope. | |
| 08:15:56 | jayaanand_ | https://www.irccloud.com/pastebin/WNKStFUI/ | |
| 08:16:17 | jayaanand_ | i am using multi-node devstack setup to troubleshoot customer issue. Devstack is deployed with latest code. during live migration i am facing below issue. your suggestion is deeply appreciate. | |
| 08:17:18 | opendevreview | Fabian Wiesel proposed openstack/nova master: Add more password generation options https://review.opendev.org/c/openstack/nova/+/865669 | |
| 08:20:38 | frickler | jayaanand_: are you using ovn or ovs? you'll likely need to check neutron logs for the cause of that port binding failure | |
| 08:24:08 | jayaanand_ | https://www.irccloud.com/pastebin/w55E78ST/ | |
| 08:24:19 | jayaanand_ | i am following link https://docs.openstack.org/devstack/latest/guides/multinode-lab.html for multi node setup. didn't perform any neutron configuration for live migration. Neutron logs has below information | |
| 08:24:50 | jayaanand_ | both ovn and ovs services are enabled in local.conf | |
| 08:36:58 | opendevreview | Fabian Wiesel proposed openstack/nova master: Add more password generation options https://review.opendev.org/c/openstack/nova/+/865669 | |
| 08:43:50 | frickler | jayaanand_: I think auniyal had similar issues following that guide, which likely is outdated. mixing ovn and ovs isn't a good idea, likely the config for the compute node needs to be updated. see also https://bugs.launchpad.net/devstack/+bug/2060802 | |
| 09:20:14 | jayaanand_ | thank you! @frickler. i really appreciate your quick response. let me try setup with OVN only | |
| 10:17:43 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.2: Functional test test_boot_reschedule_with_proper_pci_device_count https://review.opendev.org/c/openstack/nova/+/926554 | |
| 10:17:44 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.2: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926555 | |
| 14:43:43 | opendevreview | Merged openstack/nova stable/2023.2: Functional test test_boot_reschedule_with_proper_pci_device_count https://review.opendev.org/c/openstack/nova/+/926554 | |
| 14:51:31 | gibi | bauzas: Uggla: I finished the re-review of the manila series up until https://review.opendev.org/c/openstack/nova/+/871642/41 I found no new major issue. But there are open comments from the past and left couple new comments here and there to fix. I also verified locally the basic workflow with devstack (including the open manila lock fix). So far so good. I would like to test a bit more like | |
| 14:51:37 | gibi | notifications and rescue with shares. | |
| 14:53:23 | Uggla | gibi, thx for the review and feedbacks, I will have a look ASAP. | |
| 15:19:12 | mnaser | has anyone observed nova-api memory usage grow non stop | |
| 15:21:56 | mnaser | i am seeing this in zed (yeah.. it's an older environment) | |
| 15:22:10 | mnaser | using uwsgi in this case | |
| 15:25:49 | bauzas | mnaser: did you run a GMR ? | |
| 15:25:59 | bauzas | https://docs.openstack.org/nova/latest/reference/gmr.html#generating-a-gmr | |
| 15:28:24 | bauzas | in the past, we had internally memory greeding on nova-compute, we also used TCP backdoors on nova-compute to get memory details | |
| 15:28:47 | bauzas | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.backdoor_port | |
| 15:46:25 | mnaser | bauzas: i did not get a chance, i thought gmr is mostly giving us config + tracebacks i think, no? it seems it slowly grows till we kill it at 8gb | |
| 15:47:56 | bauzas | true, but you'll see all the greenthreads | |
| 15:48:13 | bauzas | my guess is that some greenthread is not closed | |
| 15:53:35 | mnaser | hmm yeah i just have another env and its like | |
| 15:53:40 | mnaser | almost 1.1g per nova-api process | |
| 15:53:56 | mnaser | https://usercontent.irccloud-cdn.com/file/F0hC2xtG/image.png | |
| 15:56:36 | mnaser | gr, uwsgi intercepts sigusr2 for its own purposes | |
| 16:07:07 | gibi | mnaser: you need to use the file trigger of GMR | |
| 16:08:49 | gibi | https://github.com/openstack/oslo.reports/blob/master/oslo_reports/opts.py#L30 | |
| 18:37:33 | opendevreview | melanie witt proposed openstack/python-novaclient master: functional: Handle multiple networks https://review.opendev.org/c/openstack/python-novaclient/+/927249 | |
| 19:07:41 | opendevreview | melanie witt proposed openstack/nova master: Adding sysctl changes for post-copy migration https://review.opendev.org/c/openstack/nova/+/908201 | |
| 19:08:18 | opendevreview | melanie witt proposed openstack/nova master: Adding sysctl changes for post-copy migration https://review.opendev.org/c/openstack/nova/+/908201 | |
| 19:22:30 | opendevreview | Merged openstack/nova master: libvirt: call get_capabilities() with all CPUs online https://review.opendev.org/c/openstack/nova/+/926218 | |
| 19:34:01 | opendevreview | Merged openstack/nova stable/2023.2: add functional repoducer for bug 2065927 https://review.opendev.org/c/openstack/nova/+/922985 | |
| 20:59:24 | opendevreview | Merged openstack/nova stable/2023.2: retry write_sys call on device busy https://review.opendev.org/c/openstack/nova/+/922986 | |
| 21:54:23 | opendevreview | Merged openstack/python-novaclient master: functional: Handle multiple networks https://review.opendev.org/c/openstack/python-novaclient/+/927249 | |
| 23:30:25 | opendevreview | sean mooney proposed openstack/nova master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/nova/+/899753 | |
| 23:43:37 | opendevreview | Merged openstack/nova stable/2023.2: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926555 | |
| 23:43:43 | opendevreview | Merged openstack/nova stable/2024.1: hardware: Correct log https://review.opendev.org/c/openstack/nova/+/927110 | |
| #openstack-nova - 2024-08-29 | |||
| 00:15:55 | opendevreview | Michael Still proposed openstack/nova master: libvirt: Add extra specs for USB redirection. https://review.opendev.org/c/openstack/nova/+/927354 | |
| 01:48:38 | melwitt | the openstack-tox-docs job appears to be failing 100% of the time which might be related to an upper-constraints bump of sphinx earlier today. I have posted to the ML https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/TSZUKN6MPOZVMQJQO7XRYIL37INH6PWM/ | |
| 01:48:42 | melwitt | cc stephenfin ^ | |
| 01:49:05 | melwitt | (sphinx expert) | |
| 03:06:38 | melwitt | stephenfin: update: I found what the problem is, sphinx 8.0.0 made overwriting of user files an error by default and the sphinx-feature-classification extension is doing an overwrite. it needs to use force=True to maintain its behavior, I have proposed a patch based on local testing https://review.opendev.org/c/openstack/sphinx-feature-classification/+/927449 | |
| 03:07:20 | melwitt | that ^ will need a version requirement on 8.0.0 or some other way to only pass force=True if >= 8.0.0. so I'm not sure how to handle that | |
| 03:09:35 | melwitt | stephenfin: I suspect it will be best to roll back the sphinx version bump in upper-constraints for now | |
| 03:35:08 | melwitt | cc bauzas ^ | |
| 05:25:30 | opendevreview | Amit Uniyal proposed openstack/nova master: VMware: updates resource provider trait list https://review.opendev.org/c/openstack/nova/+/926522 | |
| 05:25:31 | opendevreview | Amit Uniyal proposed openstack/nova master: Ironic: updates resource provider trait list https://review.opendev.org/c/openstack/nova/+/926524 | |
| 05:25:31 | opendevreview | Amit Uniyal proposed openstack/nova master: zvm: updates resource provider trait list https://review.opendev.org/c/openstack/nova/+/926523 | |
| 06:56:34 | mikal | melwitt: when is feature freeze? I'm feeling like I'm unlikely to get this patch series landed before then? | |
| 07:04:17 | mikal | Yeah, I think its this week which means this thing isn't going to happen. | |
| 07:28:32 | opendevreview | Jens Harbott proposed openstack/nova master: Delete ./doc/source/_static/support-matrix.css file https://review.opendev.org/c/openstack/nova/+/927471 | |
| 07:30:02 | frickler | melwitt: ^^ maybe this is the better solution? sphinx tries to copy .tox/docs/lib/python3.10/site-packages/sphinx_feature_classification/support-matrix.css over it anyway. at least that seemed to fix the error for me locally | |
| 07:31:41 | frickler | but if that doesn't work for some reason I have missed so far, I'm also fine with reverting the sphinx bump until next cycle | |
| 08:14:19 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.2: hardware: Correct log https://review.opendev.org/c/openstack/nova/+/927473 | |
| 08:18:50 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Functional test test_boot_reschedule_with_proper_pci_device_count https://review.opendev.org/c/openstack/nova/+/926558 | |
| 08:18:51 | opendevreview | Balazs Gibizer proposed openstack/nova stable/2023.1: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926559 | |
| 08:20:40 | bauzas | frickler: saw the email from melwitt about the sphinx update that hits the docs CI ? | |
| 08:20:54 | bauzas | oh shit, just saw your points | |
| 08:23:41 | opendevreview | Kaloyan Kotlarski proposed openstack/nova master: libvirt: Fetch the available vCPUs from the respective cgroup https://review.opendev.org/c/openstack/nova/+/927474 | |
| 08:23:43 | bauzas | frickler: do you know if others projects are impacted ? | |
| 08:36:58 | bauzas | I'm just afraid we would find other bugs so close to FF | |
| 09:14:35 | frickler | I didn't see other failures yet, but also didn't explicitly look for them so far | |
| 09:42:09 | opendevreview | Stephen Finucane proposed openstack/python-novaclient master: tox: Add Python-specific functional envs https://review.opendev.org/c/openstack/python-novaclient/+/927250 | |
| 09:42:36 | opendevreview | Stephen Finucane proposed openstack/python-novaclient master: Fix Python 3.12 compatibility https://review.opendev.org/c/openstack/python-novaclient/+/927251 | |
| 11:08:50 | opendevreview | sean mooney proposed openstack/nova master: update nova-next to use ubuntu 24.04 https://review.opendev.org/c/openstack/nova/+/922492 | |
| 12:14:17 | opendevreview | sean mooney proposed openstack/nova master: [DNM] testing uc cap https://review.opendev.org/c/openstack/nova/+/927500 | |
| 12:18:47 | sean-k-mooney | melwitt: i think https://review.opendev.org/c/openstack/openstackdocstheme/+/927496/3/openstackdocstheme/page_context.py will fix the openstackdocstheme | |
| 12:19:05 | sean-k-mooney | and i think https://review.opendev.org/c/openstack/requirements/+/927499 is what allowed the newer sphinx build | |
| 12:19:18 | sean-k-mooney | well the bump to 4.0.0 | |
| 12:20:26 | frickler | sean-k-mooney: sphinx is explicitly pinned in u-c. but also please check https://review.opendev.org/c/openstack/nova/+/927471 | |
| 12:21:10 | sean-k-mooney | that not the error we are seeign in the gate as far as i can see | |
| 12:21:20 | sean-k-mooney | or its not related to it | |
| 12:22:30 | frickler | sean-k-mooney: then maybe we are talking about two different issues, can you point me to one of your failures? | |
| 12:22:56 | sean-k-mooney | there is a new sphinx removal warning for sphinx 9 | |
| 12:23:13 | frickler | yes, but afaict that is non-fatal | |
| 12:23:14 | sean-k-mooney | we treat warnings as errors | |
| 12:23:24 | sean-k-mooney | so in the gate it is | |
| 12:24:24 | frickler | in the docs build? are you telling me my patch passes in check but will fail in gate pipeline? that would be a pretty bad design IMO | |
| 12:25:02 | sean-k-mooney | no | |
| 12:25:19 | sean-k-mooney | i thnk you test willl fail now that the new verison of sphinx is allwoed | |
| 12:25:29 | frickler | no, it ran after that | |
| 12:27:26 | sean-k-mooney | maybe we are exluding | |
| 12:27:28 | sean-k-mooney | /home/zuul/src/opendev.org/openstack/nova/.tox/docs/lib/python3.10/site-packages/openstackdocstheme/page_context.py:70: RemovedInSphinx90Warning: Sphinx 9 will drop support for representing paths as strings. Use "pathlib.Path" or "os.fspath" instead. | |
| 12:27:30 | sean-k-mooney | src_file = full_src_file[len(str(app.builder.env.srcdir)):].lstrip('/') | |
| 12:27:42 | auniyal | but shouldn't we just delete too, if its problametic, and deleting fixes the issue also, like what this patch is doing https://review.opendev.org/c/openstack/nova/+/927471 | |
| 12:27:46 | sean-k-mooney | form what is fatal | |
| 12:28:04 | auniyal | or it can cause any other error ? | |
| 12:28:20 | frickler | check https://ca0e8b358c93d8e83ff8-d2c9747c19735a1cd74d2b5484a5c4b3.ssl.cf5.rackcdn.com/927471/1/check/openstack-tox-docs/8034b4b/job-output.txt , using sphinx-8.0.2, lots of warning, no failure | |
| 12:30:13 | frickler | the failure is this: WARNING: Aborted attempted copy from /home/zuul/src/opendev.org/openstack/nova/.tox/docs/lib/python3.10/site-packages/sphinx_feature_classification/support-matrix.css to /home/zuul/src/opendev.org/openstack/nova/doc/build/html/_static/support-matrix.css (the destination path has existing data). [misc.copy_overwrite] | |