| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-07-23 | |||
| 14:52:26 | opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet.event.Event with threading.Event https://review.opendev.org/c/openstack/nova/+/949754 | |
| 15:53:16 | gmaan | stephenfin: fixed the doc comments, please check https://review.opendev.org/c/openstack/nova/+/953063 | |
| 15:54:32 | gibi | sean-k-mooney: yeah we finished in 13 minutes the agenda doc has the details what we discussed | |
| 16:15:28 | sean-k-mooney | gibi: no worries, i didnt have time to look at the work since last week anyway | |
| 16:15:40 | sean-k-mooney | gibi: so i dint have anythign to dicuss | |
| 16:16:32 | gmaan | sean-k-mooney: I merged the manager role and new policy changes into single change, please check https://review.opendev.org/c/openstack/nova/+/953063 | |
| 16:17:08 | gmaan | that gives complete view of how changes looks like, thanks for that comment. | |
| 16:18:11 | sean-k-mooney | gmaan: sure. im having a hard time tryign to decice what to work on today so im happy to take a look | |
| 16:19:26 | gmaan | sean-k-mooney: thanks, I think good timing from me :) | |
| 16:21:13 | sean-k-mooney | normally i have no issue with a lack fo prioirty in my life because im pretty good at setting my own but right now im like, there are too many things i could be working on so its nice to be handed a review and say can you look at this | |
| 16:21:47 | gmaan | ++ | |
| 16:23:20 | gibi | sean-k-mooney: OK cool | |
| 17:14:28 | opendevreview | Merged openstack/nova master: libvirt: Add objects and notifications for sound model. https://review.opendev.org/c/openstack/nova/+/926126 | |
| 17:30:01 | sean-k-mooney | gmaan: so i need ot look at the testign of the poicy change tomorrow but lookign at the code change i think that is pretty good. i left one of two nit comments but nothign that need to be changed | |
| 17:30:39 | gmaan | sean-k-mooney: ack, I have linked tempest change in same topic | |
| 17:30:39 | sean-k-mooney | gmaan: you already have tempest coverage for this too correct. ill try and look at that too tomorow to get a full picture of the testing | |
| 17:30:48 | gmaan | sure, thanks | |
| 17:54:45 | opendevreview | sean mooney proposed openstack/nova master: [DNM] testing using debian for ceph jobs https://review.opendev.org/c/openstack/nova/+/955716 | |
| #openstack-nova - 2025-07-24 | |||
| 00:45:04 | mikal | stephenfin: I can't find an example in the nova codebase of a unit test which tests parsing for a non-string flavor extra spec. I'll do one up, but I think its a thing we've not deeply tested in the past unless every other extra spec is always a string or something. | |
| 05:57:30 | jssfr | sean-k-mooney, regarding NVRAM, we're currently working on reviving https://review.opendev.org/c/openstack/nova/+/621646, too | |
| 05:57:52 | jssfr | (this is somewhat re discussion in https://review.opendev.org/c/openstack/nova/+/955657, but also because I saw your other change) | |
| 06:11:07 | opendevreview | Merged openstack/nova master: api: Add response body schemas for removed APIs https://review.opendev.org/c/openstack/nova/+/952282 | |
| 08:32:27 | opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | |
| 08:59:10 | opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | |
| 09:03:01 | opendevreview | Ian Watson proposed openstack/nova master: Implementing get_num_instances for ironic virt driver https://review.opendev.org/c/openstack/nova/+/955685 | |
| 09:16:23 | opendevreview | Ian Watson proposed openstack/nova master: Don't remove instance from cache if 409 hit https://review.opendev.org/c/openstack/nova/+/955686 | |
| 09:23:38 | opendevreview | Ian Watson proposed openstack/nova master: Changing how placement updates are triggered in RT https://review.opendev.org/c/openstack/nova/+/955687 | |
| 09:28:20 | opendevreview | Ian Watson proposed openstack/nova master: Add migrations_supported methed for resource tracker https://review.opendev.org/c/openstack/nova/+/955690 | |
| 10:22:27 | rpittau | sorry for the wall of text, hoping to find someone who can help have a look here, thanks! | |
| 10:22:27 | rpittau | it looks like the issue is somehow related to nova, at least from the errors we can see there, like 2025-07-23 21:16:59.494 | ERROR nova.objects.instance [[01;36mNone req-c824579b-82f9-499c-8b59-382bdf74f47b [00;36mNone None] [01;35m[instance: 00000000-0000-0000-0000-000000000000] Unable to migrate instance because host None with node None not found[00m: nova.exception.ComputeHostNotFound: Compute host None could not be found. | |
| 10:22:27 | rpittau | we've started to see an error in the ironic grenade job a couple of days ago, you can see an example report here https://e33cfc4e5ae5ade32027-ad9677b8f3b079990c4951ac9cbbd797.ssl.cf2.rackcdn.com/openstack/2f8dfc2059784e6691e13e08876ef108/controller/logs/grenade.sh_log.txt | |
| 10:22:27 | rpittau | hello everyone! :) | |
| 11:18:16 | gibi | rpittau: was the instance being migrated here is booted successfully on a node before? Does the test tries to migrate with a target host? If so what is the target host the test sends in the API request? | |
| 11:18:27 | gibi | does ironic supports migration at all? | |
| 11:18:28 | gibi | :) | |
| 11:42:29 | opendevreview | Lajos Katona proposed openstack/nova master: blueprint: iothreads-for-instances https://review.opendev.org/c/openstack/nova/+/939254 | |
| 12:50:12 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 12:57:47 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 12:58:14 | noonedeadpunk | sean-k-mooney: this is basically parallel migration thing we've talked last week ^ | |
| 12:58:25 | noonedeadpunk | just need to poke at unit tests a little bit | |
| 12:59:26 | noonedeadpunk | really interesting thing, is that tunneled migrations does have better performance then native migrations with tls | |
| 13:00:00 | noonedeadpunk | so the bottleneck was always not the tunnel all that time... | |
| 13:00:24 | sean-k-mooney | you mean tunneled migration with multiple connections | |
| 13:00:32 | sean-k-mooney | is beter then netive with 1? | |
| 13:00:45 | noonedeadpunk | tunneled with single is better then native with single | |
| 13:00:49 | sean-k-mooney | or is native tls with multipel connections somehow slower then tunnled with multiple | |
| 13:00:55 | noonedeadpunk | I don't think you can do parallel with tunnel | |
| 13:00:57 | sean-k-mooney | really... | |
| 13:01:08 | sean-k-mooney | that does not track with the testing we did at the tiem but if that is ture | |
| 13:01:17 | sean-k-mooney | it defintly soemthing we shoudl follow up on | |
| 13:01:20 | sean-k-mooney | to understand why | |
| 13:01:29 | noonedeadpunk | at least according to what I tested in a lab with EPYC 9534 as a cpu | |
| 13:01:41 | sean-k-mooney | that a very interesting datapoint | |
| 13:01:56 | noonedeadpunk | without tls it's way faster though | |
| 13:02:12 | noonedeadpunk | needless to say about parallel thing | |
| 13:02:57 | noonedeadpunk | I put some results here: https://review.opendev.org/c/openstack/nova-specs/+/955783/1/specs/libvirt-parallel-migrate.rst | |
| 13:06:42 | noonedeadpunk | Under tls enabled/disabled I mean [libvirt]/live_migration_with_native_tls just in case | |
| 13:07:06 | noonedeadpunk | as [libvirt]/live_migration_scheme was always `tls` | |
| 13:08:02 | noonedeadpunk | as first option is responsible for how memory is being transferred during migration by qemu, the second is how libvirt connection/auth is performed in libvirt | |
| 13:09:52 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 13:35:09 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova-specs master: Propose enabling parallel live migrations for libvirt https://review.opendev.org/c/openstack/nova-specs/+/955783 | |
| 13:54:32 | opendevreview | Balazs Gibizer proposed openstack/nova master: Document native threading mode and tuneables https://review.opendev.org/c/openstack/nova/+/949364 | |
| 13:54:33 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | |
| 13:54:33 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow services to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | |
| 13:54:34 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run nova-api and -metadata in threaded mode https://review.opendev.org/c/openstack/nova/+/951957 | |
| 13:54:34 | opendevreview | Balazs Gibizer proposed openstack/nova master: Do not yield in threading mode https://review.opendev.org/c/openstack/nova/+/950994 | |
| 13:54:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Allow to start unit test without eventlet https://review.opendev.org/c/openstack/nova/+/953436 | |
| 13:54:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Warn on long task wait time for executor https://review.opendev.org/c/openstack/nova/+/952666 | |
| 13:54:36 | opendevreview | Balazs Gibizer proposed openstack/nova master: Run unit test with threading mode https://review.opendev.org/c/openstack/nova/+/953475 | |
| 13:54:37 | opendevreview | Balazs Gibizer proposed openstack/nova master: [CI]Make nova-tox-py312-threading voting https://review.opendev.org/c/openstack/nova/+/955791 | |
| 13:54:37 | opendevreview | Balazs Gibizer proposed openstack/nova master: [test]RPC using threading or eventlet selectively https://review.opendev.org/c/openstack/nova/+/953815 | |
| 14:03:13 | opendevreview | Artem Vasilyev proposed openstack/nova master: mypy: fix type annotations for PciDevicePoolList and NUMATopologyLimits https://review.opendev.org/c/openstack/nova/+/955792 | |
| 14:09:04 | opendevreview | Artem Vasilyev proposed openstack/nova master: Fix functional tests and mypy linter on macOS https://review.opendev.org/c/openstack/nova/+/937727 | |
| 14:19:48 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 14:24:41 | opendevreview | Merged openstack/nova master: Implement sound model extra spec for libvirt. https://review.opendev.org/c/openstack/nova/+/940770 | |
| 14:42:44 | sean-k-mooney | noonedeadpunk:i did a quick pass but ya overall that in line whith what we discussed | |
| 14:42:50 | sean-k-mooney | i think that woudl be a good improvement | |
| 14:47:59 | noonedeadpunk | damn, I missed the directory for spec /o\ | |
| 15:00:13 | sean-k-mooney | what matters is you have the content in the spec that is needed to get input early and agree the direction | |
| 15:00:23 | noonedeadpunk | ++ | |
| 15:10:53 | opendevreview | Dmitriy Rabotyagov proposed openstack/nova master: [WIP] Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784 | |
| 15:11:01 | noonedeadpunk | there is also a some implementation of it in ^ | |
| 15:11:21 | noonedeadpunk | I got your comments and will adjust things accordingly | |
| 16:24:40 | stephenfin | gibi: I've been discussing this with sean-k-mooney already, but I'm seeing an issue where I get a timeout "[w]aiting for libvirt event about the detach of device" | |
| 16:26:11 | stephenfin | I'm seeing this on Dalmatian. I haven't tried later releases. It doesn't happen on Bobcat. Curiously, it also doesn't happen if I upgrade Bobcat to Dalmatian (bump all services, bump all dependencies in the global venv, compare all service configuration, ...) | |
| 16:27:44 | stephenfin | Distro is identical (22.04 with latest updates), distro packages are effectively identical, OpenStack configuration should be identical (per above), so I'm mostly stumped. I'm *also* discussing with eharney_, but I'm noting here since iirc you wrote that code initially and you may have some tidbit nestled away in the back of your head :D | |
| 16:28:37 | stephenfin | and the net effect is volume detaches take 20 seconds every single time on Dalmatian, while they happen immediately on Bobcat (upgraded to Dalmatian) | |
| 17:07:20 | opendevreview | Biser Milanov proposed openstack/nova-specs master: Add the handling-nova-compute-restarts-during-live-migration spec https://review.opendev.org/c/openstack/nova-specs/+/955812 | |
| 17:12:02 | sp-bmilanov | gibi: dansmith: based on your suggestion https://review.opendev.org/c/openstack/nova-specs/+/955812 I have to go now, but I hope we discuss it again on the next Nova weekly | |
| 17:39:15 | sean-k-mooney | ... | |
| 17:39:22 | sean-k-mooney | how is https://review.opendev.org/c/openstack/nova/+/944054 not merged already.... | |
| 17:39:43 | sean-k-mooney | this was ment to be in epoxy | |
| 17:39:51 | sean-k-mooney | before we released ... | |
| 17:57:55 | opendevreview | sean mooney proposed openstack/nova master: [tests] Add printing of sample and template paths https://review.opendev.org/c/openstack/nova/+/944054 | |
| 17:57:55 | opendevreview | sean mooney proposed openstack/nova master: only show standard image properties in server show. https://review.opendev.org/c/openstack/nova/+/942413 | |
| 17:59:24 | sean-k-mooney | dansmith: can you readd your +2 to those... that was ment to be included before we released epoxy. i cosidered that to be a release blocker so sin tthat aprpently didnt happen im keen to get that fixed in stable as soon as possible | |
| 18:00:47 | sean-k-mooney | i only noticed that was not there because i went to debug a test an notice my follow patch to improve the test loging was missing | |
| 18:04:28 | gibi | sean-k-mooney: fast approved both | |
| 18:04:42 | sean-k-mooney | thansk | |
| 18:05:07 | sean-k-mooney | i think we just missed this in the scamble to fix all the tempest schemam issues that those micoverions caused | |