| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-08-05 | |||
| 11:32:29 | mohsen__ | I have set the following configs in my nova compute nova.conf file: | |
| 11:32:29 | mohsen__ | hello everyone. I have an issue with resource tracking in openstack. | |
| 11:32:30 | mohsen__ | The issue is that the placement service periodically adds the specified set of dedicated cpus to the inventory as PCPU section, and after a while, it removes the PCPU section of the inventory unexpectedly. which results to not being able to delete instances and getting the following error: | |
| 11:32:30 | mohsen__ | and I enabled the NUMATopologyFilter and AggregateInstanceExtraSpecsFilter filters so that I can schedule cpu-pinned instances on my desired aggregate of compute hosts. | |
| 11:32:32 | mohsen__ | Error during ComputeManager.update_available_resource: nova.exception.ReshapeNeeded: Virt driver indicates that provider inventories need to be moved. | |
| 11:32:34 | mohsen__ | I appreciate your ideas in advance | |
| 11:57:24 | sean-k-mooney | mohsen__: is this an existign deployment with workload? | |
| 11:57:53 | sean-k-mooney | modifyign the legacy vcpu_pin_set and newer cpu_*_sets shoudl only be done on empty hosts | |
| 12:00:28 | sean-k-mooney | im not aware of a bug that would cause the issue you describe but it sound like the "reshape" code that is run when upgrading to a release that supprot pinned and unpined guests on the same host is trying to fix the allocations/inventories. i dont know what woudl cause the pcpu invetoies to be removed other hten perhaps if the resouce tracker was broken by vms running on the | |
| 12:00:30 | sean-k-mooney | wrong cores. that can happen if you defiend or modifed those values when vms were on the hosts | |
| 12:02:42 | sean-k-mooney | darkhackernc: if the compute service is down the instance is not expected to be manageable via the nova api | |
| 12:06:14 | mohsen__ | Thank you for your helpful response | |
| 12:06:14 | mohsen__ | sean-k-mooney: Yes. My OpenStack environment already has a couple of instances up and running on it. | |
| 12:13:16 | sean-k-mooney | darkhackernc: this looks like a masikari issue so i marked the bug as invliad for nova. | |
| 12:26:13 | NicolaiRuckel | jkulik: That really helped. I was able to fix the problem now. | |
| 12:31:37 | jkulik | NicolaiRuckel: nice! | |
| 12:36:21 | opendevreview | Lajos Katona proposed openstack/nova master: WIP Use SDK for Neutron https://review.opendev.org/c/openstack/nova/+/928022 | |
| 13:09:25 | mohsen__ | sean-k-mooney: I migrated all the instances from one compute node to another computes. then reconfigured nova with the mentioned config above on the compute host where there aren't any instances no longer. then I ran "openstack resource provider inventory list <compute-host-uuid>" command and find out that the same issue still exists. what could be the root cause? | |
| 13:10:34 | sean-k-mooney | mohsen__: you have restared the compute agent on the host since it was drained? | |
| 13:11:49 | mohsen__ | sean-k-mooney: since I used kolla-ansible to reconfigure it, it has a handler which do the restart task after modification. I didn't do it manually but I can test it. | |
| 13:12:01 | sean-k-mooney | if so and the issue still happens i think your going to have to file a bug with some logs. i.e. the full error message or warning form the reouce tracker and or perodic task related to the createion or update of the resouce provider | |
| 13:12:44 | sean-k-mooney | mohsen__: you did the reconfiugre while there were isntances on the host correct | |
| 13:12:54 | sean-k-mooney | have you restarted it since it was empty | |
| 13:14:25 | mohsen__ | sean-k-mooney: I evacuated all the instances and then ran the kolla-ansible to reconfigure it. the restart task had been run after all the instances were evacuated. | |
| 13:18:04 | mohsen__ | sean-k-mooney: After evacuating all the instances and after reconfiguration, I manually restarted it and the issue still persist. sounds like there is no way unless reporting a bug🤔 | |
| 13:18:15 | sean-k-mooney | ack, without inspecigng the logs and or error i cant really advise more. i likely wont have time to dig much deeper in the short term unfortunetly. i woudl suggeset filing a bug and trying to extract any relevent log messages (ideally at debug level) that are related to teh execution of the update_aviable_resouce provider perodic and the intiall startup of the agent related to | |
| 13:18:17 | sean-k-mooney | when it initally created teh RP invetories with the PCPUs inventory. you mentioned it initally was created and then lost correct? it would be good to try and capture the relevent logs showing both | |
| 13:21:16 | mohsen__ | sean-k-mooney: Thank you for your guidance | |
| 14:58:27 | cardoe | I know the nova meeting is coming up and I just want to ping on https://review.opendev.org/c/openstack/nova-specs/+/471815 I pinged earlier. | |
| 15:30:42 | Uggla | Nova meeting in ~30mn | |
| 16:01:39 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:01:39 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:01:39 | opendevmeet | Meeting started Tue Aug 5 16:01:39 2025 UTC and is due to finish in 60 minutes. The chair is Uggla. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:01:39 | Uggla | #startmeeting nova | |
| 16:01:51 | Uggla | Hello everyone | |
| 16:02:02 | jssfr | o/ | |
| 16:02:04 | jssfr | NicolaiRuckel and me would like to bring up a topic in the "Open Discussion" slot | |
| 16:02:08 | jssfr | (jfyi) | |
| 16:02:29 | fwiesel | o/ | |
| 16:02:44 | Uggla | jssfr, no pb but this week few people are available. | |
| 16:03:01 | jssfr | thanks for the heads up :) | |
| 16:03:08 | gmaan | o/ | |
| 16:04:04 | Uggla | awaiting people to join, but I'm pretty sure we will not have quorum. | |
| 16:04:36 | Uggla | Please raise your hand for showing you are part of the meeting. | |
| 16:05:07 | NicolaiRuckel | o/ | |
| 16:05:40 | dansmith | o/ | |
| 16:06:03 | Uggla | Let's start | |
| 16:06:09 | Uggla | #topic Bugs (stuck/critical) | |
| 16:06:16 | Uggla | #info No Critical bug | |
| 16:06:58 | melwitt | o/ | |
| 16:07:04 | Uggla | #topic Gate status | |
| 16:07:14 | Uggla | hey melwitt, good to see you | |
| 16:07:23 | Uggla | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:07:29 | Uggla | #link https://etherpad.opendev.org/p/nova-ci-failures-minimal | |
| 16:07:36 | Uggla | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status | |
| 16:07:48 | Uggla | #info Please look at the gate failures and file a bug report with the gate-failure tag. | |
| 16:07:58 | Uggla | #info Please try to provide a meaningful comment when you recheck | |
| 16:08:06 | Uggla | #info Gate was blocked last week with "openstack.exceptions.SDKException: Image creation failed: 'latin-1' codec can't encode character '\u2019' in position 662: ordinal not in range(256) in openstack.tests.functional.image.v2.test_image.TestImage.test_tags". | |
| 16:08:13 | Uggla | #info https://review.opendev.org/c/openstack/openstacksdk/+/956369 fixed it | |
| 16:08:49 | Uggla | #topic tempest-with-latest-microversion job status | |
| 16:08:58 | Uggla | #link https://zuul.opendev.org/t/openstack/builds?job_name=tempest-with-latest-microversion&skip=0 | |
| 16:09:10 | Uggla | gmaan, do you have something to share ? | |
| 16:10:28 | gmaan | ah, no update on this. | |
| 16:10:47 | Uggla | thanks, so moving on | |
| 16:10:55 | Uggla | #topic Release Planning | |
| 16:11:02 | Uggla | #link https://releases.openstack.org/flamingo/schedule.html | |
| 16:11:07 | Uggla | #info Nova deadlines are set in the above schedule | |
| 16:12:15 | Uggla | Time is flying, a bit less than 4 weeks before FF. | |
| 16:12:26 | Uggla | #topic Review priorities | |
| 16:12:36 | Uggla | #link https://etherpad.opendev.org/p/nova-2025.2-status | |
| 16:13:12 | Uggla | I guess the file is updated, please let me know if something is wrong in it. | |
| 16:13:29 | Uggla | #topic OpenAPI | |
| 16:13:37 | Uggla | #link: https://review.opendev.org/q/topic:%22openapi%22+(project:openstack/nova+OR+project:openstack/placement)+-status:merged+-status:abandoned | |
| 16:13:59 | Uggla | #info up from 13 to 32 remaining. (I guess Stephen created new ones and follow up) | |
| 16:14:24 | Uggla | I hope we can see an end about this. | |
| 16:15:01 | Uggla | I'm going to skip stable branch topic because Elod is on pto. | |
| 16:15:28 | Uggla | #topic vmwareapi 3rd-party CI efforts Highlights | |
| 16:15:36 | Uggla | fwiesel, something on our side ? | |
| 16:15:44 | fwiesel | #info fwiesel will be on PTO until the 9/9 | |
| 16:15:54 | fwiesel | Otherwise, no news from me. | |
| 16:16:07 | fwiesel | Uggla: back to you | |
| 16:16:23 | Uggla | fwiesel, I wish you'll enjoy your PTO. | |
| 16:16:33 | fwiesel | Thanks! | |
| 16:17:25 | Uggla | Also skiping Gibi's point about eventlet removal due to Gibi's well deserved pto. | |
| 16:17:59 | Uggla | #topic Open discussion | |
| 16:18:17 | melwitt | I have a small topic | |
| 16:18:18 | Uggla | So jssfr, giving the mic to you. | |
| 16:18:26 | jssfr | thanks! | |
| 16:18:38 | jssfr | so we're currently looking into two topics which are somewhat related: | |
| 16:18:41 | jssfr | first we're rebasing https://review.opendev.org/c/openstack/nova/+/621646 | |
| 16:18:53 | jssfr | second I proposed a draft patch for a vTPM data preservation issue a couple weeks ago: https://review.opendev.org/c/openstack/nova/+/955657 | |
| 16:19:22 | sean-k-mooney | o/ sorry was doign something else | |
| 16:19:26 | jssfr | o/ | |
| 16:19:38 | jssfr | I also think that sean-k-mooney proposed something similar to the vTPM draft for UEFI NVRAM preservation | |
| 16:20:00 | jssfr | we'd like to hear whether we're heading the right direction by rebasing #621646 regarding UEFI, or whether there might be a better way. | |
| 16:20:17 | sean-k-mooney | i was suggeing bring it up in this meetign yes | |
| 16:20:20 | melwitt | oh, good. this was the topic I wanted heh (vTPM data) | |
| 16:20:24 | sean-k-mooney | or rather that melwitt bring it up | |
| 16:20:35 | jssfr | I also saw only now that you replied to #955657, because I had my inbox sorting messed up, reading that now. | |
| 16:21:00 | sean-k-mooney | melwitt: do you want to summerize | |