Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2026-08-19
09:40:24 gibi so about scsi. On my machine the VM is created successfully but cannot boot. It stops at the EFI setup not seeing a boot device
09:56:28 antia that is consistent with my previous experience
10:27:15 sean-k-mooney that is becaus of the qemu firmware image yes?
10:27:25 sean-k-mooney so it may or may not work dependin on that
10:27:41 sean-k-mooney im not sure if we shoudl relaly block it in nova but we shoudl call it out in the docs
10:39:43 gibi yepp call it out in the doc
10:40:06 gibi (and yes I tested without TDX that the same image boots from scsi)
10:40:58 sean-k-mooney ack, i partly wonder if we are jsut mising setting somethign liek the iommu options we recnelty stop setting for sev
10:41:15 sean-k-mooney but i guess we can follow up with improvment next cycle if we discover a way to supprot it
10:41:22 sean-k-mooney and jsut treat it as a minor bugfix
10:41:41 gibi OK
11:16:23 opendevreview Bhavana proposed openstack/nova-specs master: Add spec for MIG-aware PCI Host Weigher https://review.opendev.org/c/openstack/nova-specs/+/998753
11:52:49 sean-k-mooney im generally not sure ^ is something that is really in nova scope now
11:53:20 sean-k-mooney cybrog will need a way to provide its own way to experss schdulign prioties in the future
11:53:31 sean-k-mooney be it a weigher or somethign more generic in nova
11:54:11 sean-k-mooney but if we are actully intending this fucntionatly to go out of nova then addign mig aware ness to nova is questionable
11:54:32 sean-k-mooney nova can mange vgpus on PGUS configure with mig
11:54:41 sean-k-mooney but its is not aware mig is in use
12:35:14 opendevreview Balazs Gibizer proposed openstack/nova master: [DNM]: Troubleshoot race condition in unshelve vif plug https://review.opendev.org/c/openstack/nova/+/1001294
12:35:15 opendevreview Balazs Gibizer proposed openstack/nova master: [WIP]Add task_state = None as pre-condition of unshelve https://review.opendev.org/c/openstack/nova/+/1001485
12:35:31 gibi gmaan: ^^ I have hunch based on your last comment
12:36:00 sean-k-mooney unshivlign while still shevle offloading?
12:36:04 gibi yepp
12:36:11 gibi it seems like we have a race
12:36:25 gibi the offloading removes the instance.host while unshelve is already claimed
12:36:27 sean-k-mooney ya i can see that being a problem
12:36:50 sean-k-mooney also ya addign task state to the decoreato rmakes seesne
12:36:58 gibi and it might happen more often in the grenade job as it uses a single conductor setup instead of a super and a cell conductor
12:37:19 gibi so the two instance.save happens in the same conductor
12:37:37 sean-k-mooney maybe although i dont think that would make it that much slower with onloy 1-2 vms
12:37:43 sean-k-mooney *1-2 computes
12:37:48 gibi yeah, I'm just guessing here
12:37:58 sean-k-mooney i think the race hunch is a good one
12:38:09 sean-k-mooney jus tnot entirly sure why its more common in grenade
12:38:10 gibi we will see. if the precondition makes the tempest test fail early that will prove it
12:38:47 gibi it can be something with threading + single conductor as well. I think only grenade is doing that now
12:39:00 sean-k-mooney is the tempest tst properly waiting for it to be shleve offloaded or just unshilving straight away
12:39:05 gibi threading allows more random task switching
12:39:27 gibi sean-k-mooney: even if the tempest waits for the shelve_offload state we might set that early with a task state
12:39:30 gibi let me check
12:39:49 sean-k-mooney gibi: well im thinking that tempest shoudl retry the unshleve
12:40:03 sean-k-mooney at least on a 409
12:40:10 sean-k-mooney which is what that decoreator shoudl be returning
12:40:58 sean-k-mooney since the transtion form shelve to shelve offloaded is configurabel and depend on if its BFV or BFI
12:41:14 sean-k-mooney or using ceph
12:41:28 gibi https://github.com/openstack/tempest/blob/fa340b1f5fff8f57a018be206644c81b6d36b3d4/tempest/common/compute.py#L388-L400
12:41:30 sean-k-mooney so i dont think we can write the test to know which to expect
12:41:37 gibi so it depends on the tempest config what we are waiting for
12:41:42 sean-k-mooney but it can retyr the unshelve on 409 conflict and still be correct
12:42:10 gibi I can imagine we end up only waiting for SHELVED state and issue an unshelve
12:42:19 sean-k-mooney yep
12:42:37 sean-k-mooney but as i said for images_type=rbd and bfv we go drictly to shelve_offloaded i think
12:42:41 gibi OK we will see and if it is the case then we need to fix nova and tempest as well
12:42:43 sean-k-mooney regradless fo the offload time
12:43:33 sean-k-mooney ya we should have ci reulst in a while and we can see
12:45:53 gibi yepp
14:15:09 dansmith gibi: I just +2'd this if you want to re-add yours after the test coverage fix: https://review.opendev.org/c/openstack/nova/+/998571
14:23:28 gibi dansmith: ohh sure. Thanks for the reviews there
14:30:00 gibi Uggla: I left some feedback in the repro and fix in https://review.opendev.org/c/openstack/nova/+/999978
14:30:33 gibi I agree this is a valid repro but for the fix we might need to make a step back and see if we can make a wider fix
14:30:56 gibi having an instance without task state and without instance.host during a period of evac sounds a source of different issuees
14:45:05 gibi Uggla: on the same note I will need to drop at 17:45 latest from the upstream triage call today
14:47:37 opendevreview Balazs Gibizer proposed openstack/nova master: [DNM]: Troubleshoot race condition in unshelve vif plug https://review.opendev.org/c/openstack/nova/+/1001294
14:47:38 opendevreview Balazs Gibizer proposed openstack/nova master: [WIP]Add task_state = None as pre-condition of unshelve https://review.opendev.org/c/openstack/nova/+/1001485
14:47:56 gibi grr ^^ that was in merge conflict while I thought I just waiting for the CI result
15:29:51 Uggla Upstream triage meeting
15:29:52 Uggla meet.google.com/zjr-rxus-hzj
15:48:42 gmaan gibi: thanks, I was thinking to do some check on conductor side but this is better
15:49:56 gmaan gibi: sambork sorry i missed eventlet meeting today. if anything for me to review, please let me know.
15:52:05 sambork np it was mostly about perf test
15:55:45 gmaan ack
16:50:57 opendevreview Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211
17:11:10 opendevreview Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211
18:01:58 melwitt gmaan: can you remind me if there is a fix idea for https://bugs.launchpad.net/nova/+bug/2160254 ? anecdotally I'm seeing a lot of failures on patches I'm following
18:02:50 gmaan melwitt: not yet, I did not debug it yet. most probably I will be able to get on this after FF
18:03:00 gmaan is it happening too frequently ?
18:03:45 melwitt seems so but that is just my personal observation on patches I'm following. I guess I should check opensearch
18:05:18 gmaan k, let me see if I can get some time today. it may get worst during FF week :)
18:06:21 melwitt gmaan: looks like it's just my bad luck, I see only one hit per day at most
18:07:43 gmaan ok, I used to see it mulitple times in the week when i reported bug but agfter that not much occurance, but thanks for reminder, i will see if there is any obious mistake in test
19:17:32 gmaan gibi: i saw notification failure in functional threading job, it seems 'start' migration event happened instead of 'end' https://zuul.opendev.org/t/openstack/build/07c0f0aecebb42fc99ec209ad6a9df3e/log/job-output.txt#13762
20:56:47 opendevreview Ghanshyam Maan proposed openstack/nova master: Create services required thread pool executors during service initialization https://review.opendev.org/c/openstack/nova/+/1001572
21:01:04 gmaan dansmith: gibi: ^^ this is the further refactoring of thread pool executors as discussed in original change
21:20:16 opendevreview Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211
#openstack-nova - 2026-08-20
03:03:02 opendevreview Ghanshyam Maan proposed openstack/nova master: Shutdown thread pool executors during graceful shutdown https://review.opendev.org/c/openstack/nova/+/1000211
03:57:00 opendevreview Ghanshyam Maan proposed openstack/nova master: Create services required thread pool executors during service initialization https://review.opendev.org/c/openstack/nova/+/1001572
04:00:55 gmaan gibi: i saw the same notification test failing again, reported the bug with trackback and link https://bugs.launchpad.net/nova/+bug/2164606
04:01:29 opendevreview Ghanshyam Maan proposed openstack/nova master: Create services required thread pool executors during service initialization https://review.opendev.org/c/openstack/nova/+/1001572
13:38:36 dansmith gouthamr: I've restarted m-shr and no reconcile has happened. I've captured the logs for the service during the restart, a dump of the manila database, and a capture of what osc shows me
13:38:51 dansmith gouthamr: anything else you want me to capture (other service logs) before I force fix the share and move on?
14:21:38 gouthamr dansmith: o/ ty, looking
14:57:04 opendevreview Brian Haley proposed openstack/nova master: Change neutron API calls to use project_id https://review.opendev.org/c/openstack/nova/+/994321
15:22:17 gouthamr dansmith: in meetings and got distracted; did you have the log posted somewhere? i can take a look at it and i'm thinking we have an eventlet bug.. I don't need anyhing else though, so yes.. force-fix the share with "access delete --unrestrict" as admin
15:22:50 dansmith gouthamr: no, I was waiting to see if you wanted me to capture anything else.. I'll post what I have in a sec then
15:23:01 gouthamr dansmith: i reported this https://bugs.launchpad.net/manila/+bug/2164609 and started investigating.. my hunch is this has to do with the eventlet backend (default today)
15:23:31 gouthamr dansmith: tyty
15:23:51 dansmith gouthamr: the logs and things could go in the bug.. do you want the DB dump there as well (or not at all?)
15:26:17 dansmith oh, is that bug for something more specific and thus not quite appropriate?
15:26:29 dansmith FWIW, it does seem like restarting the service takes a long time, like it's waiting for something
15:28:18 gmaan dansmith: btw, this is ready. CI is flaky over other unrelated failure so zuul -1 https://review.opendev.org/c/openstack/nova/+/1000211/14

Earlier   Later