Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-08-27
14:26:59 gibi dansmith: I can take a look quickly now
14:27:08 sean-k-mooney i can try and make time to review it too
14:27:10 gibi unfortunately I missed it completly
14:27:15 dansmith I wish it was already because the context is out of my head now.. but yes, would very much like to have it merged
14:28:33 sean-k-mooney looks like you adressed my main comments. i just discarded one i had pienidng because its not worth changign the patch for
14:29:01 sean-k-mooney dansmith: the answer to https://review.opendev.org/c/openstack/placement/+/945464/comment/0189c002_387845d8/ was just the test is doing to diffent things so i woudl have made it two tests
14:29:38 sean-k-mooney ill just ack tthat comment and we can leave it as is
14:32:14 sean-k-mooney the repoducer looks fine ill need to spen a littel more time on the actual fix but ill do that later today
14:32:17 ralonsoh sean-k-mooney, yes, you can rely on this flag instead of the port binding
14:32:28 ralonsoh we are setting now in both places (the field and port binding)
14:32:39 ralonsoh we'll remove the last one (port binding) once accepted in Nova
14:33:36 sean-k-mooney ack. ill add nova ot the bug and check with other if we are ok to fix and backport that as a security hardening bug
14:33:55 sean-k-mooney if not then ill propose doing it as a specless blueprint next cycle
14:42:49 dansmith gibi: I was going to ping our PTL, but ... https://review.opendev.org/c/openstack/nova/+/957424
14:43:00 gibi dansmith: I'm +2 on both patch in placement about the overallocation. Thanks both was pretty clear.
14:43:01 dansmith I wonder if we should point them to a specific outlet (or outlets) to report success
14:43:07 dansmith gibi: sweet thanks
14:44:04 gibi dansmith: I can ask them to drop a mail to the ML or show up on IRC.
14:44:41 gibi probably ML is easier for them
14:44:45 dansmith yeah the former is what I was thinking.. I know some may not even go that far, so I wish we had another venue but.. at least asking for success reporting seems like a good idea
14:45:01 gibi I will respin and add it
14:45:12 gibi (and in the same time I will remove the nova-conductor from the series)
14:45:18 dansmith maybe put bauzas' personal email and cell number (as our closest TC rep) in there if they don't want to join the ML :D
14:45:30 gibi lol :)
14:45:32 bauzas hmmm, WAT ?
14:45:33 dansmith gibi: also why not just squash those renos into one?
14:45:54 bauzas $context ?
14:46:00 gibi dansmith: about the renos there. We already have two from two independent patch but yeah I can try to merge them in that commit and see what our doc gen says about a deleted reno
14:46:21 dansmith not a big deal but when I read the second one I thought it was accidental duplication until I flipped back and forth a couple times to see
14:46:33 dansmith gibi: oh right, duh, okay nevermind then
14:47:41 dansmith usually renos in patches are new and I was reading it as if you were adding those.. no need to fix now I think
14:47:50 gibi OK
15:01:08 gibi Uggla: ping :)
15:02:21 Uggla gibi pong ! :)
15:02:48 gibi probably a netspilt happened between the IRC servers but those should be reported to us. At least in the univesity days that was somewhat frequent and that was a way to steal nicks :)
15:11:59 bauzas gibi: dansmith: what was the problem with reno ? sorry, hardly following
15:12:10 dansmith yeah I think after some netsplit my nick list just didn't recover.. I restarted my client and it's back now
15:12:13 dansmith bauzas: no problem
15:12:16 bauzas also, I'd appreciate if some cores could give a look at tkajinam's series on SEV ES
15:12:53 bauzas I reviewed that, Uggla found some typos that tkajinam will update but apart from that, we're in a very good spot to support next generation of AMD SEV :)
15:12:57 dansmith bauzas: as I said, I'm not reviewing that before tomorrow.. I'm totally blank on it and don't think I'd be doing a good job in such a short time frame... just FYI
15:13:22 bauzas dansmith: well, it was a general reminder to all of us, not particularly you who already said that indeed, cool
15:13:35 dansmith yep, I know, just sayin' :)
15:14:30 bauzas there is also a quite simple feature patch worth reviewing, which is to copy our provider.yaml, I started reviewing it and I have an open question : https://review.opendev.org/c/openstack/nova/+/948304
15:14:42 bauzas we could fix that in a FUP if other cores like it
15:15:41 gmaan dansmith: RE 957578: ah it merge conflict, I will rebase it. it was all good until last night. testing and everything done for this and it is good to review (once i rebase)
15:15:48 dansmith ack
15:16:24 bauzas gmaan: I can give it a shot
15:16:40 bauzas fwiw, the etherpad is updated with my findings https://etherpad.opendev.org/p/nova-2025.2-status
15:16:50 bauzas people can use it for directing reviews
15:29:18 lajoskatona bauzas, Uggla: Hi, perhaps you can help me out, we have a bug originally from a downstream customer: https://bugs.launchpad.net/nova/+bug/2051685 for migration
15:30:22 lajoskatona bauzas, Uggla: it hit us few cycles back and than rubasov made an upstream reproduction which you can read in the bug description but he hit some walls in understanding the depths of Nova as you can read at the end of the report :-)
15:31:59 lajoskatona bauzas, Uggla: if we should have a pointer if the issue is really an issue and can be fixed would be enough help to keep working on it but we lost how to dig deeper and if it is really something that can be / worth to be fixed
15:33:27 Uggla lajoskatona, at least I can add this bug for review in our next bug scrubbing.
15:34:10 lajoskatona Uggla: Thanks for your time, if any more info is needed we can check and add o course to the LP
15:35:21 Uggla lajoskatona, I'll try to discuss it in next nova meeting. But I can't promise we will have time, depending of the agenda.
15:36:14 lajoskatona Uggla: no problem, thanks
16:15:21 sean-k-mooney lajoskatona: so use reset state after a migration error si never corect
16:15:31 sean-k-mooney so if this only happens when you do that its not really a valid bug
16:16:47 sean-k-mooney lajoskatona: also im surpesed we accepted --availability-zone :devstack0
16:17:05 sean-k-mooney in new micorversion you ment to pass --host
16:17:16 sean-k-mooney if you jsut want to slect a host
16:18:01 sean-k-mooney i tought the az was intended to be requried and just passing :<host> was not intended to be supprot but perhaps that worked by acidnet
16:18:08 lajoskatona sean-k-mooney: reset state was only used to show that in devstack the issue can happen, as it happened in some customer site without proper logs and reproduction
16:18:12 sean-k-mooney we never really encurraged that usage
16:18:50 sean-k-mooney right but reset state will break novas ablity ot properly clean up
16:19:31 sean-k-mooney as will restarting the nova-comapute durign the mighration
16:20:31 lajoskatona the issue is that when something happens with the destination host during migration the migration can stuck in error case
16:20:54 sean-k-mooney well yo say stuck in error but i tough erro was a valid end state
16:21:04 sean-k-mooney so its not stuck its in the terminal state
16:21:10 sean-k-mooney or one of the terminal states
16:21:20 lajoskatona as rubasov added to the last lines as a thought experiment: "I don't see it yet, how this should be fixed. Maybe a migration should never be stuck indefinitely in status=error...."
16:21:27 sean-k-mooney failed models a diffent endsate then error
16:22:12 sean-k-mooney error is something went wrong that we cant recover form gracefully liek the dest host gets restarted
16:22:27 lajoskatona yes of course
16:22:29 sean-k-mooney failed is the end state when it somethig we can properly recover form
16:22:38 sean-k-mooney like we try to migrate and the cpu is incompatible
16:23:13 sean-k-mooney so the migration status being in error for ever is nto a bug to me
16:23:51 sean-k-mooney the orignal issue was caused by "overcommit on dedicated PCPUs and CPUPinningInvalid exception"
16:24:23 sean-k-mooney I.E. there was already invlaid sate in one or boht of the compute nodes
16:24:52 lajoskatona and the update_available_resource periodically fetches these also and tries to apply the migration that ended in error state but the repeated migration for the same server to the same host succeded so it should be find by the method that the migration is really done by a new migration (server uuid - dest host pair)
16:24:54 sean-k-mooney i.e. vms not runningn on the right core or the oepration modfied the nova config to change the core set
16:26:41 lajoskatona so the question is if we can say that ahhh, the same VM to the same host migration operation is finished so we can forget to stress on the previous migration (same vm uuid same host) that failed ?
16:28:13 sean-k-mooney in general no
16:28:17 lajoskatona so it is really a corner case as I see not error due to the things you mentioned (pcpu config or such) but like issue with the compute itself and after the fix the migration can be repeated
16:28:36 sean-k-mooney but we liekly do have a bug in the logic
16:29:00 sean-k-mooney i dont thnk it woudl be correct for the erroed migration to be updated to failed
16:29:06 sean-k-mooney is shoudl stay in error
16:29:27 sean-k-mooney but if the vm is not on this host we shoudl not process the old migration on new iterations
16:29:39 sean-k-mooney of the update_aviable_resouces perodic
16:29:49 sean-k-mooney movign it to fialed might achive that
16:29:54 lajoskatona that's fair to rethink it and make it clear and we can document it for customers in such case in worst case
16:30:07 sean-k-mooney but im not sure we can alwasy assume that is safe to do
16:30:29 lajoskatona sounds great to think at least about it
16:31:10 sean-k-mooney you were testing bfv was it using ceph by the way?
16:31:48 sean-k-mooney if so the already exist erro might eb related to https://docs.openstack.org/nova/latest/configuration/config.html#workarounds.ensure_libvirt_rbd_instance_dir_cleanup
16:31:54 lajoskatona to tell the truth I am not sure but would say no (have to double check)
16:32:53 sean-k-mooney https://bugs.launchpad.net/nova/+bug/1414895
16:32:59 sean-k-mooney is one of the rrefenece bugs
16:33:59 sean-k-mooney we dont generally delete the instnace storage on the dest on start up because we dont want ot loose data

Earlier   Later