| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2026-02-25 | |||
| 22:49:37 | opendevreview | melanie witt proposed openstack/nova master: TPM: support live migration of `deployment` secret security https://review.opendev.org/c/openstack/nova/+/925771 | |
| 22:49:37 | opendevreview | melanie witt proposed openstack/nova master: TPM: test live migration between hosts with different security https://review.opendev.org/c/openstack/nova/+/952629 | |
| 22:49:38 | opendevreview | melanie witt proposed openstack/nova master: TPM: add late check for supported TPM secret security https://review.opendev.org/c/openstack/nova/+/956975 | |
| 22:49:38 | opendevreview | melanie witt proposed openstack/nova master: TPM: enable conversion of secret security modes via resize https://review.opendev.org/c/openstack/nova/+/962052 | |
| 22:49:39 | opendevreview | melanie witt proposed openstack/nova master: DNM vtpm tempest https://review.opendev.org/c/openstack/nova/+/957477 | |
| 23:54:42 | cardoe | gmaan: neat. So we'll be able to shutdown gracefully and let it continue to manage things. | |
| 23:59:03 | gmaan | cardoe: yes, we introduced the 2 RPC server in compute so that compute stop accepting the new request and finish the in-progress one. in this phase-1, services wait for configuration timeout which will be replaced by task tracking in next part. | |
| #openstack-nova - 2026-02-26 | |||
| 00:05:12 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add operator document for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977590 | |
| 01:33:05 | opendevreview | Ghanshyam Maan proposed openstack/nova master: Add operator document for graceful shutdown https://review.opendev.org/c/openstack/nova/+/977590 | |
| 01:35:56 | gmaan | gate is in its worst situation | |
| 02:38:41 | gmaan | gibi: sean-k-mooney and this doc also need re-review, fixed pep8 spell things https://review.opendev.org/c/openstack/nova/+/977590 | |
| 03:00:11 | opendevreview | Merged openstack/nova master: api: Add response body schemas for servers APIs (6/6) https://review.opendev.org/c/openstack/nova/+/956240 | |
| 08:50:26 | opendevreview | Merged openstack/nova master: api: Add response body schemas for server shares APIs https://review.opendev.org/c/openstack/nova/+/956266 | |
| 08:50:39 | opendevreview | Merged openstack/nova master: tests: Invert validation check https://review.opendev.org/c/openstack/nova/+/956241 | |
| 08:59:22 | gibi | I'm envy about the folks who can land patches through the gate. The eventlet series now stuck for more than 24 hours behind random failures. I feel bad about grinding rechecks but I will do it anyhow :/ | |
| 09:15:37 | frickler | let me know if you need help like re-enqueueing changes directly back into gate or even force-merging. once you're sufficiently sure the failures are unrelated IMO it doesn't make sense to keep doing rechecks and using up CI resources | |
| 09:16:11 | frickler | and I say this knowing that getting tests more stable is a high priority for the nova team anyway | |
| 09:18:51 | gibi | yeah, we need to stabilize our gate | |
| 09:19:08 | gibi | and thanks I might eventually ask for help requeuing | |
| 09:28:56 | sean-k-mooney | frickler: gibi os we can remove a class of error form the ceph jobs it we just move them to useing nested virt (kvm) | |
| 09:29:12 | sean-k-mooney | nova doesnt do that but nuetorn and some other teams do | |
| 09:30:03 | sean-k-mooney | for exampel the kernel panics on volume detach | |
| 09:31:00 | sean-k-mooney | the ssh timeout might go away if we enabel the creation fo the tap devices by os-vif. i need to try enableing that in the gate now that the neutron bits are all in place | |
| 09:32:23 | sean-k-mooney | but yes we take some time to adress more of those failures. the issue nova has is those failure are often not form nova/hypervior comonents | |
| 09:32:34 | sean-k-mooney | i.e. the etc/ceph contianer download fialures | |
| 09:33:16 | sean-k-mooney | *etcd | |
| 09:33:46 | sean-k-mooney | i think we can improve ^ in devstack by stragicly retrying some fo the downloads | |
| 09:33:51 | bauzas | do we have at least an idea on why the failures are more recurrent those days ? those look like usual suspects | |
| 09:34:36 | bauzas | (speaking of the volume detach kernel panics and the ssh timeouts - the latter already being WA'd a lot) | |
| 09:34:47 | sean-k-mooney | bauzas: i dont think any of the failure gibi has been hitting were new. they all looked like the "ususal suspects" | |
| 09:35:13 | sean-k-mooney | so my guess is load make it manifest more | |
| 09:35:29 | sean-k-mooney | the kernel panics dont happen if we use either kvm or the uec images | |
| 09:35:52 | sean-k-mooney | both of which change (speedup) the timing of the kernel loading in cirrus | |
| 09:36:06 | gibi | yeah load does not help | |
| 09:36:18 | gibi | I just saw a guest bootop where every step took a second | |
| 09:36:20 | gibi | SEND | |
| 09:36:30 | gibi | [ 4.017889] clocksource: Switched to clocksource tsc | |
| 09:36:30 | gibi | [ 5.258926] ACPI: \_SB_.LNKB: Enabled at IRQ 10 | |
| 09:36:30 | gibi | [ 5.553620] Freeing initrd memory: 8864K | |
| 09:36:30 | gibi | [ 6.239466] ACPI: \_SB_.LNKC: Enabled at IRQ 11 | |
| 09:36:30 | gibi | [ 7.080979] ACPI: \_SB_.LNKD: Enabled at IRQ 11 | |
| 09:36:33 | gibi | [ 7.966979] ACPI: \_SB_.LNKA: Enabled at IRQ 10 | |
| 09:36:35 | gibi | [ 8.834095] Free page reporting enabled | |
| 09:36:38 | gibi | [ 8.839828] Serial: 8250/16550 driver, 32 p | |
| 09:36:52 | sean-k-mooney | ya... | |
| 09:36:54 | gibi | this is not sustainable with our current ssh timeouts | |
| 09:38:20 | sean-k-mooney | so the ceph jobs i think we shoudl consdier movign to the nested virt lables to compenstate for the ceph cpu load or look at the perfromence of them in general because those are the ones that hit those time out most often | |
| 09:38:29 | sean-k-mooney | and the kernel panincs most often | |
| 09:38:57 | sean-k-mooney | it would be intersting to compare the boot time form one of those jobs vs the non ceph jobs | |
| 09:39:06 | sean-k-mooney | or look at the load average for the jobs | |
| 09:40:04 | gibi | eah | |
| 09:40:06 | gibi | yeah | |
| 09:40:18 | sean-k-mooney | gibi: i take it the eventlet changes didnt merge last night | |
| 09:51:12 | sean-k-mooney | amazed that any of the jobs pass? :) | |
| 09:51:46 | bauzas | gibi: were you able to see the vmstat values of the host at the time the VM was booting that fast ? | |
| 09:53:20 | bauzas | sean-k-mooney: looks like we play a Claw machine game | |
| 11:07:27 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary action method prefix https://review.opendev.org/c/openstack/nova/+/958863 | |
| 11:07:28 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Deprecate os-volumes_boot API https://review.opendev.org/c/openstack/nova/+/958864 | |
| 11:09:54 | opendevreview | Stephen Finucane proposed openstack/nova master: conf: Deprecate AggregateImagePropertiesIsolation opts https://review.opendev.org/c/openstack/nova/+/783396 | |
| 11:09:55 | opendevreview | Stephen Finucane proposed openstack/nova master: conf: Deprecate AggregateImagePropertiesIsolation opts https://review.opendev.org/c/openstack/nova/+/783396 | |
| 11:28:13 | opendevreview | Stephen Finucane proposed openstack/nova master: docs: Update REST API version history for 2.101 https://review.opendev.org/c/openstack/nova/+/978002 | |
| 11:28:13 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Fix typo https://review.opendev.org/c/openstack/nova/+/958855 | |
| 11:28:14 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Add runtime check for query additionalProperties https://review.opendev.org/c/openstack/nova/+/958856 | |
| 11:28:14 | opendevreview | Stephen Finucane proposed openstack/nova master: tests: Add missing test coverage https://review.opendev.org/c/openstack/nova/+/958857 | |
| 12:14:31 | opendevreview | sean mooney proposed openstack/nova master: ci: Define nested nodesets, audit KVM/UEC, move nova-next https://review.opendev.org/c/openstack/nova/+/978066 | |
| 12:21:49 | gibi | sean-k-mooney: it didn't. I still rechecking :/ | |
| 12:22:27 | gibi | bauzas: I did not dig deeper. I see overall slowness so it is probably infra overload due to FF churn | |
| 12:22:28 | sean-k-mooney | ^ might help but it has not run in ci yet to see if its functionatl | |
| 12:22:41 | sean-k-mooney | lol nope | |
| 12:22:51 | sean-k-mooney | it has zuul sytax errors | |
| 12:22:56 | sean-k-mooney | which i thinkis a typo | |
| 12:34:53 | opendevreview | sean mooney proposed openstack/nova master: ci: Define nested nodesets, audit KVM/UEC, move nova-next https://review.opendev.org/c/openstack/nova/+/978066 | |
| 12:37:14 | sean-k-mooney | was missign a 1 in compute1 | |
| 12:47:52 | opendevreview | sean mooney proposed openstack/nova master: ci: Define nested nodesets, audit KVM/UEC, move nova-next https://review.opendev.org/c/openstack/nova/+/978066 | |
| 13:00:37 | opendevreview | sean mooney proposed openstack/nova master: ci: Define nested nodesets, audit KVM/UEC, move nova-next https://review.opendev.org/c/openstack/nova/+/978066 | |
| 13:00:52 | sean-k-mooney | i shoudl not try to update patches while also in an irc meeting... | |
| 13:25:53 | gmaan | oh, i hoped at least a few will be merged overnight... | |
| 13:26:10 | opendevreview | Merged openstack/nova master: api: Fix issue with instance usage audit log schema https://review.opendev.org/c/openstack/nova/+/977967 | |
| 13:26:27 | opendevreview | Merged openstack/nova master: api: Add runtime check for general additionalProperties https://review.opendev.org/c/openstack/nova/+/958858 | |
| 13:26:44 | opendevreview | Merged openstack/nova stable/2024.2: api: Fix validators for hw:cpu_max_* extra specs https://review.opendev.org/c/openstack/nova/+/957744 | |
| 13:26:57 | sean-k-mooney | so it si posibel to merge things ^ | |
| 13:27:06 | sean-k-mooney | but that seriese is apprently cursed | |
| 13:27:19 | DominikDanelski[m] | sean-k-mooney: In https://review.opendev.org/c/openstack/nova/+/971176/comment/e96c41d6_d900c91a/ you mentioned 'repurposing' the spec. Is it done by essentially making a modified copy that targets a newer release and has no direct link to the previous one, or should git move the accepted one from Ussuri, since it wasn't implemented then, modify it, and add an appropriate line to redirects file? | |
| 13:28:07 | sean-k-mooney | DominikDanelski[m]: yes you copy the spec to the new folder and update the history section at the end to note its re proposed | |
| 13:28:47 | sean-k-mooney | we do not move them as we want to keep every revsion fo the spec that were prevsouly approved so we can clearly see any deltas | |
| 13:29:23 | DominikDanelski[m] | All right, thank you. | |
| 13:49:26 | opendevreview | Merged openstack/nova master: libvirt: Remove unnecessary arg https://review.opendev.org/c/openstack/nova/+/881817 | |
| 14:16:07 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary action method prefix https://review.opendev.org/c/openstack/nova/+/958863 | |
| 14:16:08 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Deprecate os-volumes_boot API https://review.opendev.org/c/openstack/nova/+/958864 | |
| 14:19:32 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary action method prefix https://review.opendev.org/c/openstack/nova/+/958863 | |
| 14:19:32 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Deprecate os-volumes_boot API https://review.opendev.org/c/openstack/nova/+/958864 | |
| 14:35:17 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Remove unnecessary action method prefix https://review.opendev.org/c/openstack/nova/+/958863 | |
| 14:35:17 | opendevreview | Stephen Finucane proposed openstack/nova master: api: Deprecate os-volumes_boot API https://review.opendev.org/c/openstack/nova/+/958864 | |
| 14:42:41 | gibi | dansmith: o/ I'm +2 on the making the volume attach async in https://review.opendev.org/c/openstack/nova/+/971068 | |
| 14:43:08 | dansmith | ack | |
| 14:50:08 | cardoe | sean-k-mooney: around? Wanted to follow up about what I posted here. | |
| 14:51:13 | Uggla | gibi, bauzas, any chance we can merge https://review.opendev.org/q/topic:%22bp/generalize-sev-code%22 to my mind it is ok except maybe the latest one. | |
| 14:53:21 | bauzas | Uggla: I was trying to look at it | |
| 14:54:15 | Uggla | dansmith it would be super cool if you can check https://review.opendev.org/c/openstack/nova/+/971068 | |
| 14:54:16 | gibi | Uggla: I have couple of +2s in the series | |