| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-07-24 | |||
| 06:11:07 | opendevreview | Merged openstack/nova master: api: Add response body schemas for removed APIs https://review.opendev.org/c/openstack/nova/+/952282 | |
| 08:32:27 | opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | |
| 08:59:10 | opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | |
| 09:03:01 | opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | |
| 09:16:23 | opendevreview | Ian Watson proposed openstack/nova master: Don't remove instance from cache if 409 hit https://review.opendev.org/c/openstack/nova/+/955686 | |
| 09:23:38 | opendevreview | Ian Watson proposed openstack/nova master: Changing how placement updates are triggered in RT https://review.opendev.org/c/openstack/nova/+/955687 | |
| 09:28:20 | opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | |
| 10:22:27 | rpittau | sorry for the wall of text, hoping to find someone who can help have a look here, thanks! | |
| 10:22:27 | rpittau | it looks like the issue is somehow related to nova, at least from the errors we can see there, like 2025-07-23 21:16:59.494 | ERROR nova.objects.instance [[01;36mNone req-c824579b-82f9-499c-8b59-382bdf74f47b [00;36mNone None] [01;35m[instance: 00000000-0000-0000-0000-000000000000] Unable to migrate instance because host None with node None not found[00m: nova.exception.ComputeHostNotFound: Compute host None could not be found. | |
| 10:22:27 | rpittau | we've started to see an error in the ironic grenade job a couple of days ago, you can see an example report here https://e33cfc4e5ae5ade32027-ad9677b8f3b079990c4951ac9cbbd797.ssl.cf2.rackcdn.com/openstack/2f8dfc2059784e6691e13e08876ef108/controller/logs/grenade.sh_log.txt | |
| 10:22:27 | rpittau | hello everyone! :) | |
| 11:18:16 | gibi | rpittau: was the instance being migrated here is booted successfully on a node before? Does the test tries to migrate with a target host? If so what is the target host the test sends in the API request? | |
| 11:18:27 | gibi | does ironic supports migration at all? | |
| 11:18:28 | gibi | :) | |
| 11:42:29 | opendevreview | Lajos Katona proposed openstack/nova master: blueprint: iothreads-for-instances https://review.opendev.org/c/openstack/nova/+/939254 | |
| 12:50:12 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 12:57:47 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 12:58:14 | noonedeadpunk | sean-k-mooney: this is basically parallel migration thing we've talked last week ^ | |
| 12:58:25 | noonedeadpunk | just need to poke at unit tests a little bit | |
| 12:59:26 | noonedeadpunk | really interesting thing, is that tunneled migrations does have better performance then native migrations with tls | |
| 13:00:00 | noonedeadpunk | so the bottleneck was always not the tunnel all that time... | |
| 13:00:24 | sean-k-mooney | you mean tunneled migration with multiple connections | |
| 13:00:32 | sean-k-mooney | is beter then netive with 1? | |
| 13:00:45 | noonedeadpunk | tunneled with single is better then native with single | |
| 13:00:49 | sean-k-mooney | or is native tls with multipel connections somehow slower then tunnled with multiple | |
| 13:00:55 | noonedeadpunk | I don't think you can do parallel with tunnel | |
| 13:00:57 | sean-k-mooney | really... | |
| 13:01:08 | sean-k-mooney | that does not track with the testing we did at the tiem but if that is ture | |
| 13:01:17 | sean-k-mooney | it defintly soemthing we shoudl follow up on | |
| 13:01:20 | sean-k-mooney | to understand why | |
| 13:01:29 | noonedeadpunk | at least according to what I tested in a lab with EPYC 9534 as a cpu | |
| 13:01:41 | sean-k-mooney | that a very interesting datapoint | |
| 13:01:56 | noonedeadpunk | without tls it's way faster though | |
| 13:02:12 | noonedeadpunk | needless to say about parallel thing | |
| 13:02:57 | noonedeadpunk | I put some results here: https://review.opendev.org/c/openstack/nova-specs/+/955783/1/specs/libvirt-parallel-migrate.rst | |
| 13:06:42 | noonedeadpunk | Under tls enabled/disabled I mean [libvirt]/live_migration_with_native_tls just in case | |
| 13:07:06 | noonedeadpunk | as [libvirt]/live_migration_scheme was always `tls` | |
| 13:08:02 | noonedeadpunk | as first option is responsible for how memory is being transferred during migration by qemu, the second is how libvirt connection/auth is performed in libvirt | |
| 13:09:52 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 13:35:09 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 13:54:32 | opendevreview | Balazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables https://review.opendev.org/c/openstack/nova/+/949364 | |
| 13:54:33 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | |
| 13:54:33 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow services to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | |
| 13:54:34 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode https://review.opendev.org/c/openstack/nova/+/951957 | |
| 13:54:34 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | |
| 13:54:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet https://review.opendev.org/c/openstack/nova/+/953436 | |
| 13:54:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor https://review.opendev.org/c/openstack/nova/+/952666 | |
| 13:54:36 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run unit test with threading mode https://review.opendev.org/c/openstack/nova/+/953475 | |
| 13:54:37 | opendevreview | Balazs Gibizer proposed openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | |
| 13:54:37 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively https://review.opendev.org/c/openstack/nova/+/953815 | |
| 14:03:13 | opendevreview | Artem Vasilyev proposed openstack/nova master: mypy: fix type annotations for PciDevicePoolList and NUMATopologyLimits https://review.opendev.org/c/openstack/nova/+/955792 | |
| 14:09:04 | opendevreview | Artem Vasilyev proposed openstack/nova master: Fix functional tests and mypy linter on macOS https://review.opendev.org/c/openstack/nova/+/937727 | |
| 14:19:48 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 14:24:41 | opendevreview | Merged openstack/nova master: Implement sound model extra spec for libvirt. https://review.opendev.org/c/openstack/nova/+/940770 | |
| 14:42:44 | sean-k-mooney | noonedeadpunk:i did a quick pass but ya overall that in line whith what we discussed | |
| 14:42:50 | sean-k-mooney | i think that woudl be a good improvement | |
| 14:47:59 | noonedeadpunk | damn, I missed the directory for spec /o\ | |
| 15:00:13 | sean-k-mooney | what matters is you have the content in the spec that is needed to get input early and agree the direction | |
| 15:00:23 | noonedeadpunk | ++ | |
| 15:10:53 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 15:11:01 | noonedeadpunk | there is also a some implementation of it in ^ | |
| 15:11:21 | noonedeadpunk | I got your comments and will adjust things accordingly | |
| 16:24:40 | stephenfin | gibi: I've been discussing this with sean-k-mooney already, but I'm seeing an issue where I get a timeout "[w]aiting for libvirt event about the detach of device" | |
| 16:26:11 | stephenfin | I'm seeing this on Dalmatian. I haven't tried later releases. It doesn't happen on Bobcat. Curiously, it also doesn't happen if I upgrade Bobcat to Dalmatian (bump all services, bump all dependencies in the global venv, compare all service configuration, ...) | |
| 16:27:44 | stephenfin | Distro is identical (22.04 with latest updates), distro packages are effectively identical, OpenStack configuration should be identical (per above), so I'm mostly stumped. I'm *also* discussing with eharney_, but I'm noting here since iirc you wrote that code initially and you may have some tidbit nestled away in the back of your head :D | |
| 16:28:37 | stephenfin | and the net effect is volume detaches take 20 seconds every single time on Dalmatian, while they happen immediately on Bobcat (upgraded to Dalmatian) | |
| 17:07:20 | opendevreview | Biser Milanov proposed openstack/nova-specs master: Add the handling-nova-compute-restarts-during-live-migration spec https://review.opendev.org/c/openstack/nova-specs/+/955812 | |
| 17:12:02 | sp-bmilanov | gibi: dansmith: based on your suggestion https://review.opendev.org/c/openstack/nova-specs/+/955812 I have to go now, but I hope we discuss it again on the next Nova weekly | |
| 17:39:15 | sean-k-mooney | ... | |
| 17:39:22 | sean-k-mooney | how is https://review.opendev.org/c/openstack/nova/+/944054 not merged already.... | |
| 17:39:43 | sean-k-mooney | this was ment to be in epoxy | |
| 17:39:51 | sean-k-mooney | before we released ... | |
| 17:57:55 | opendevreview | sean mooney proposed openstack/nova master: [tests] Add printing of sample and template paths https://review.opendev.org/c/openstack/nova/+/944054 | |
| 17:57:55 | opendevreview | sean mooney proposed openstack/nova master: only show standard image properties in server show. https://review.opendev.org/c/openstack/nova/+/942413 | |
| 17:59:24 | sean-k-mooney | dansmith: can you readd your +2 to those... that was ment to be included before we released epoxy. i cosidered that to be a release blocker so sin tthat aprpently didnt happen im keen to get that fixed in stable as soon as possible | |
| 18:00:47 | sean-k-mooney | i only noticed that was not there because i went to debug a test an notice my follow patch to improve the test loging was missing | |
| 18:04:28 | gibi | sean-k-mooney: fast approved both | |
| 18:04:42 | sean-k-mooney | thansk | |
| 18:05:07 | sean-k-mooney | i think we just missed this in the scamble to fix all the tempest schemam issues that those micoverions caused | |
| 18:05:13 | sean-k-mooney | i.e. missed that it failed to merge | |
| 18:05:33 | sean-k-mooney | i was 100% certian this was in | |
| 18:14:18 | gmaan | tempest sche3ma check key-value pattern not exact values so we are good there https://github.com/openstack/tempest/blob/master/tempest/lib/api_schema/response/compute/v2_98/servers.py#L32 | |
| 18:15:13 | sean-k-mooney | gmaan: ya the tempest side is fine | |
| 18:15:35 | sean-k-mooney | it was more that we discoverd the issue when trying to merge the speice stuff and hadn several patches in flight | |
| 18:15:43 | sean-k-mooney | this just fell through the cracks | |
| 18:16:45 | sean-k-mooney | gmaan: by the way i didnt find time to look at the testing of the manager role stuff today but ill try and do that tomorrow | |
| 18:17:22 | gmaan | sean-k-mooney: no issue. thanks | |
| 20:19:31 | opendevreview | Merged openstack/nova master: api: Add response body schemas for networks API https://review.opendev.org/c/openstack/nova/+/952283 | |
| #openstack-nova - 2025-07-25 | |||
| 02:13:56 | opendevreview | melanie witt proposed openstack/nova master: Add hw:tpm_secret_security extra spec validation https://review.opendev.org/c/openstack/nova/+/940197 | |
| 02:13:56 | opendevreview | melanie witt proposed openstack/nova master: Add hw_tpm_secret_security image property https://review.opendev.org/c/openstack/nova/+/940196 | |
| 02:13:57 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `user` secret security https://review.opendev.org/c/openstack/nova/+/942502 | |
| 02:13:57 | opendevreview | melanie witt proposed openstack/nova master: TPM: migrate legacy instances to new security policy https://review.opendev.org/c/openstack/nova/+/942501 | |
| 02:13:58 | opendevreview | melanie witt proposed openstack/nova master: Add vtpm_secret_(uuid|value) to LiveMigrateData https://review.opendev.org/c/openstack/nova/+/952628 | |
| 02:13:58 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `host` secret security https://review.opendev.org/c/openstack/nova/+/941795 | |
| 02:13:59 | opendevreview | melanie witt proposed openstack/nova master: TPM: support instances with `deployment` secret security https://review.opendev.org/c/openstack/nova/+/942021 | |
| 02:13:59 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `host` secret security https://review.opendev.org/c/openstack/nova/+/941483 | |
| 02:14:00 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `deployment` secret security https://review.opendev.org/c/openstack/nova/+/925771 | |
| 02:14:01 | opendevreview | melanie witt proposed openstack/nova master: TPM: update instance request_spec with secret security https://review.opendev.org/c/openstack/nova/+/952630 | |
| 02:14:01 | opendevreview | melanie witt proposed openstack/nova master: TPM: test live migration between hosts with different security https://review.opendev.org/c/openstack/nova/+/952629 | |
| 02:14:03 | opendevreview | melanie witt proposed openstack/nova master: TPM: confirm secret security via hard reboot https://review.opendev.org/c/openstack/nova/+/955847 | |