| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-09-25 | |||
| 21:09:59 | opendevreview | melanie witt proposed openstack/nova master: DNM vtpm tempest https://review.opendev.org/c/openstack/nova/+/957477 | |
| #openstack-nova - 2025-09-26 | |||
| 06:01:58 | LarsErikP | melwitt: correct, but it failed creation because of the error I described | |
| 06:03:10 | melwitt | LarsErikP: yeah, I think you have found a bug, unless this is some known issue that is documented somewhere. I did not find anything indicating that though | |
| 06:03:31 | LarsErikP | Which I suspect happened because of the "require_image_format_match". Glance somehow interprets the "dummy"/empty image as raw, when nova creates it as qcow2 | |
| 06:07:29 | LarsErikP | melwitt: I'll report a bug to glance then =) | |
| 06:07:32 | melwitt | LarsErikP: right. glance used to not validate that the formats match (it is a relatively recent development) so in the past it went through without an error but now that glance is validating by default, it reveals the mismatch. nova does not actually set anything in the disk_format field AFAICT, it leaves it empty and maybe glance defaults to qcow2 if empty (I could be wrong). and as you found you can work around it by disabling the | |
| 06:07:32 | melwitt | format validation. I'm not sure what/how we could fix this but it seems like a bug to me or at least if it's a known issue it should be documented somewhere | |
| 06:08:15 | LarsErikP | yeah. I'll file a bug, and describe it as good as I can. I guess that'll help troubleshooting, and pin-pointing what's happening | |
| 06:08:36 | melwitt | LarsErikP: it might be considered a nova bug. I'm not sure if there is a practical way nova could check the format and send the right one to glance. if we can do that then that would be the right fix I think | |
| 06:12:07 | LarsErikP | Good point. Either that, or there is something that can be fixed in the format validation itself. I don't know enough details about how this is done. It may be worth mentioning that I tried to set the "snapshot_image_format" option in nova.conf to raw in all relevant nova-servers as well. Did not change the behaviour | |
| 06:12:53 | LarsErikP | Which led me to belive that nova is just hardcoded to upload that "dummy" image as qcow2. I did not search in nova-code to confirm that though | |
| 06:13:54 | melwitt | yeah, same. this is why I think nova isn't sending any disk_format to glance https://github.com/openstack/nova/blob/b99a882366251f88d145e27312b94692e0b2266f/nova/compute/api.py#L3465-L3466 | |
| 06:16:17 | LarsErikP | ah. good catch. It's definitely not hardcoding qcow2 there ;) | |
| 06:16:45 | LarsErikP | so.. do you think I should file the bug to nova or glance then? | |
| 06:17:28 | melwitt | I'm thinking nova. we'll discuss it first and if we think it should be glance we can change it to glance | |
| 06:18:05 | melwitt | I think nova is supposed to be sending the right format but it's not so clear when it's the volume-backed case where it involves the weird dummy image | |
| 06:18:49 | LarsErikP | right. Something to chew on for the developers then.. I'll finnish my morning coffee and write up a bug report =) | |
| 06:19:34 | melwitt | cool, thanks | |
| 06:58:59 | LarsErikP | melwitt: https://bugs.launchpad.net/nova/+bug/2125730 =) | |
| 08:18:03 | opendevreview | Arnaud Morin proposed openstack/nova master: Fix debug PciPassthroughFilter https://review.opendev.org/c/openstack/nova/+/907976 | |
| 09:46:50 | opendevreview | Lajos Katona proposed openstack/nova master: Use SDK for Neutron extensions https://review.opendev.org/c/openstack/nova/+/962270 | |
| 12:02:25 | gibi | dansmith: this is a small fix for the devstack fake-virt support due to stable-compute-uuid prevented the fake nova-compute restarts https://review.opendev.org/c/openstack/devstack/+/962300 | |
| 13:30:02 | dansmith | gibi: would you rather just make it like ironic and not have to have the same rules? | |
| 13:35:55 | gibi | dnaI tried that but | |
| 13:36:15 | gibi | dansmith: I tried that but our own functional testing depends on the fake driver to test the compute_id logic | |
| 13:38:39 | dansmith | ah okay | |
| #openstack-nova - 2025-09-27 | |||
| 03:06:23 | opendevreview | melanie witt proposed openstack/nova master: Opt-in to new TPM secret security via resize https://review.opendev.org/c/openstack/nova/+/962052 | |
| 03:06:24 | opendevreview | melanie witt proposed openstack/nova master: FUP for vTPM live migration https://review.opendev.org/c/openstack/nova/+/962309 | |
| 03:06:24 | opendevreview | melanie witt proposed openstack/nova master: DNM vtpm tempest https://review.opendev.org/c/openstack/nova/+/957477 | |
| 16:22:45 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Leave chown operation of tpm data directory to libvirt https://review.opendev.org/c/openstack/nova/+/962446 | |
| 16:23:27 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Drop redundant chown of tpm data directory https://review.opendev.org/c/openstack/nova/+/962446 | |
| 16:43:29 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Drop redundant chown of tpm data directory https://review.opendev.org/c/openstack/nova/+/962446 | |
| 23:36:51 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Drop redundant chown of tpm data directory https://review.opendev.org/c/openstack/nova/+/962446 | |
| #openstack-nova - 2025-09-28 | |||
| 14:15:10 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 15:27:16 | opendevreview | Takashi Kajinami proposed openstack/nova master: libvirt: Drop redundant chown of tpm data directory https://review.opendev.org/c/openstack/nova/+/962446 | |
| 15:29:44 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 16:13:45 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 17:06:50 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 18:38:30 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 19:28:35 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| #openstack-nova - 2025-09-29 | |||
| 08:20:20 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 08:46:59 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 09:43:31 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 10:00:07 | opendevreview | Jonas Schäfer proposed openstack/nova master: Preserve vTPM state between power off and power on https://review.opendev.org/c/openstack/nova/+/955657 | |
| 10:11:49 | opendevreview | Kamil Sambor proposed openstack/nova master: Switch nova-conductor to use global executor https://review.opendev.org/c/openstack/nova/+/962478 | |
| 11:12:10 | opendevreview | Jonas Schäfer proposed openstack/nova master: Preserve vTPM state between power off and power on https://review.opendev.org/c/openstack/nova/+/955657 | |
| 11:28:28 | opendevreview | Balazs Gibizer proposed openstack/nova master: Rally job for eventlet-removal https://review.opendev.org/c/openstack/nova/+/960130 | |
| 12:14:16 | gibi | dansmith: fyi we can catch threading slowdowns now with rally https://review.opendev.org/c/openstack/nova/+/960130/30#message-1f12ccedc335aa0e59d469a478012a41d76fd67c | |
| 12:14:33 | gibi | (this was a simulated slowdown by injecting sleep to nova) | |
| 12:40:42 | opendevreview | Kamil Sambor proposed openstack/nova master: Switch nova-conductor to use global executor https://review.opendev.org/c/openstack/nova/+/962478 | |
| 13:09:06 | opendevreview | Merged openstack/nova stable/2025.1: Regression test for ImagePropsWeigher due to missing context https://review.opendev.org/c/openstack/nova/+/961805 | |
| 13:58:58 | dansmith | gibi: nice.. hopefully that will be stable enough to be useful.. even within a single worker performance has varied a lot in the past (due to noisy neighbors) but even if we get a smoke signal that will be good | |
| 14:04:47 | gibi | if the performance of the worker fluctuates in a ~10 mintues window then yeah we will get false postive signals | |
| 14:05:20 | gibi | the window size depends one length of our rally scenarios | |
| 14:17:39 | sean-k-mooney-pto | while we could have noise neighber effects i think its a lease worth trying this for a while and see if it becomes a problem | |
| 14:18:15 | sean-k-mooney-pto | gibi: presumable the rally job wont be voting at least not in the short term | |
| 14:18:32 | gibi | won't be voting for a while | |
| 14:18:35 | sean-k-mooney-pto | so if it a perodic or its non voting | |
| 14:18:43 | sean-k-mooney-pto | i think this ok for now | |
| 14:19:19 | sean-k-mooney-pto | gibi: do you think it need to run in check longterm or would you jsut rung this in weekly perodic? | |
| 14:23:29 | gibi | right now it is a cheap job so I would run it non-voting. But if we add more scenarios (I planning to ) then it might be expensive enough to run it only periodically | |
| 14:23:55 | gibi | or if we move it to a label with many subnodes | |
| 14:24:11 | sean-k-mooney-pto | ack | |
| 14:24:47 | sean-k-mooney-pto | this is the kind fo job where it would be nice if there as a persitent mysql db where we coudl log some data across runs | |
| 14:25:12 | sean-k-mooney-pto | i.e. to track over all trends across time | |
| 14:26:24 | sean-k-mooney-pto | but ya i was thining the same start we non voting and hten move to perodic + experimental if its becomes expensive | |
| 14:27:12 | sean-k-mooney-pto | havign it in experminetal so you can spot check specific changes is ueful if we go the perodic route | |
| 14:29:45 | sean-k-mooney | gibi: the last day before i went on pto i spend a little time tryign to see if we could get any better infro form atop or the logs on overall load. i made some progress but the binary format and way that tool works is a littel painful | |
| 14:40:14 | gibi | sean-k-mooney: I actually noticed that when a service is stopped systemd already nicely logs the memory swap high watermark and the cpu time of the service | |
| 14:40:20 | gibi | so I went with that | |
| 14:40:32 | sean-k-mooney | yes i saw that also | |
| 14:40:52 | sean-k-mooney | however we only have the logs form the second run correct | |
| 14:41:02 | sean-k-mooney | unless thats also in the journal dump? | |
| 14:41:56 | gibi | we have journal logs from both runs, and I do an extra service restart now at the end to get the resource usage after the second run as well. Then I pull the data out from the journal into eg https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_eb1/openstack/eb1c16166a9e42b5be0987f5d8ca0d01/results/resource-usage-devstack%40n-sch.service.txt | |
| 14:43:23 | sean-k-mooney | ah cool | |
| 14:46:00 | jssfr | dansmith (and maybe others following https://review.opendev.org/c/openstack/nova/+/955657 ): I'm going to be AFK starting october 1st until november 25th. Is there anything else I need to do before that to get it over the line? | |
| 14:47:31 | dansmith | jssfr: sorry, just been really busy with other stuff.. I think it mostly looks good to me now but I was just going to do some tests locally that I haven't had a chance to do yet | |
| 14:47:45 | dansmith | but others may feel more comfortable just looking at it | |
| 14:48:25 | jssfr | related question: is it possible to take over another person's gerrit patch? | |
| 14:48:34 | jssfr | I might find a substitute who can continue to look into it while I'm away. | |
| 14:48:56 | dansmith | yep, easy and common | |
| 14:50:05 | jssfr | is there anything that needs to be taken into account when doing that? or can another gerrit user just take my patch and run `git review`? | |
| 14:50:36 | opendevreview | Merged openstack/nova stable/2025.1: Add admin context when filling metadata on ImagePropsWeigher https://review.opendev.org/c/openstack/nova/+/961806 | |
| 14:50:40 | dansmith | no, anyone can git review it right now | |
| 14:50:59 | jssfr | thanks | |
| 14:51:09 | clarkb | `git review -d changenumber` make edits then `git review` to push back | |
| 16:48:48 | bauzas | dansmith: given we have yet another issue with the ImageMetaWeigher https://bugs.launchpad.net/nova/+bug/2125935 I wonder why I can't just target a cell context for fill_metadata this way : | |
| 16:49:03 | bauzas | with nova_context.target_cell(ctxt, cell_mapping): | |
| 16:49:03 | bauzas | insts.fill_metadata() | |
| 16:49:24 | bauzas | I guess I need to change the object context itself, right? | |
| 16:49:25 | dansmith | well, that's targeting a context but you're not using it | |
| 16:49:57 | dansmith | all the instances are from the same cell right? If so, you need to target the context, then pass that to InstanceList.__init__() and then fill metadata | |
| 16:50:19 | dansmith | the usage is "with target_cell(...) as child_ctx: " | |
| 16:52:00 | bauzas | yeah, given we look at all the instances from the same host, they are on the same context | |
| 16:52:07 | bauzas | s/context/cell | |
| 16:52:54 | bauzas | gotcha, I got that | |
| 16:53:00 | bauzas | thanks | |
| 16:53:00 | dansmith | cool, yeah then just need to target the one you give to the InstanceList | |
| 17:00:41 | bauzas | with nova_context.target_cell(ctxt, cell_mapping) as cell_ctxt: | |
| 17:00:41 | bauzas | insts = objects.InstanceList(cell_ctxt, | |