Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-08-04
13:11:35 opendevreview Dmitriy Rabotyagov proposed openstack/nova master: Allow to perform parallel live migrations https://review.opendev.org/c/openstack/nova/+/955784
13:49:58 NicolaiRuckel I tried to implement some changes in Nova's libvirt.driver and now some test cases fail with "nova.exception.ObjectActionError: Object action obj_load_attr failed because: attribute system_metadata not lazy-loadable". Does anyone have an idea where to look for debugging? I couldn't figure out where this would be set yet.
#openstack-nova - 2025-08-05
02:56:18 cardoe Hey all. I've brought this spec up before and folks have said they're good with it. There's a number of operators that are patching the nova code with patches to achieve this. I'm trying to advocate for these downstream operators to engage more with upstream OpenStack projects with my TC member hat on. https://review.opendev.org/c/openstack/nova-specs/+/471815
02:56:48 cardoe We're now seeing associated projects (neutron) marking the patches as abandoned because they don't believe it'll go anywhere.
02:57:31 cardoe Please help me folks.
06:17:07 jkulik NicolaiRuckel: sounds to me like you're using system_metadata in your patches, but the object doesn't have that loaded - and can't load it later on. system_metadata would have to get queried out (or set, since we're talking about tests) when the Instance is fetched from the DB/created in the tests.
09:27:36 opendevreview Stephen Finucane proposed openstack/nova master: tests: Replace keystoneclient with keystoneauth1 https://review.opendev.org/c/openstack/nova/+/951744
09:27:38 NicolaiRuckel jkulik: The weird thing is that I didn't knowingly touch system_metadata at all.
09:42:20 jkulik NicolaiRuckel: then maybe your patch calls some function that does consume it? little hard to tell ;)
09:52:24 NicolaiRuckel What surprised me is that the tests (for example functional.libvirt.test_uefi:test_create_server) always had the assertion `self.assertIn('image_hw_machine_type', instance.system_metadata)` and used to pass but somehow I managed to make it not lazy-loadable anymore. Is there any documentation on how system_metadata gets set/defined?
09:53:39 jkulik afaik, there's just code.
09:54:10 NicolaiRuckel I was afraid that this would be the answer. :D
09:54:28 jssfr NicolaiRuckel, maybe you can link / paste the specific patch you're looking at?
09:59:06 NicolaiRuckel one second, I need to make the repository public first
10:05:52 NicolaiRuckel This seems to be the commit that breaks the test: https://gitlab.com/NicolaiRuckel/nova/-/commit/9d6e6f44d5129a9de3c7f4a636e82411a711e3e3
10:15:23 jkulik Your use of `objects.Intance` looks weird. If I understand it right, you're assigning the `uuid` to the class - not an individual Instance instance. Maybe that breaks stuff.
10:31:09 NicolaiRuckel That sounds reasonable. I'll look into that. Thanks!
11:24:37 darkhackernc https://bugs.launchpad.net/nova/+bug/2119126 team can someone look into this
11:32:29 mohsen__ hello everyone. I have an issue with resource tracking in openstack.
11:32:29 mohsen__ I have set the following configs in my nova compute nova.conf file:
11:32:29 mohsen__ [compute]
11:32:29 mohsen__ cpu_shared_set = 0-17,36-53
11:32:29 mohsen__ cpu_dedicated_set = 18-35,54-71
11:32:30 mohsen__ and I enabled the NUMATopologyFilter and AggregateInstanceExtraSpecsFilter filters so that I can schedule cpu-pinned instances on my desired aggregate of compute hosts.
11:32:30 mohsen__ The issue is that the placement service periodically adds the specified set of dedicated cpus to the inventory as PCPU section, and after a while, it removes the PCPU section of the inventory unexpectedly. which results to not being able to delete instances and getting the following error:
11:32:32 mohsen__ Error during ComputeManager.update_available_resource: nova.exception.ReshapeNeeded: Virt driver indicates that provider inventories need to be moved.
11:32:34 mohsen__ I appreciate your ideas in advance
11:57:24 sean-k-mooney mohsen__: is this an existign deployment with workload?
11:57:53 sean-k-mooney modifyign the legacy vcpu_pin_set and newer cpu_*_sets shoudl only be done on empty hosts
12:00:28 sean-k-mooney im not aware of a bug that would cause the issue you describe but it sound like the "reshape" code that is run when upgrading to a release that supprot pinned and unpined guests on the same host is trying to fix the allocations/inventories. i dont know what woudl cause the pcpu invetoies to be removed other hten perhaps if the resouce tracker was broken by vms running on the
12:00:30 sean-k-mooney wrong cores. that can happen if you defiend or modifed those values when vms were on the hosts
12:02:42 sean-k-mooney darkhackernc: if the compute service is down the instance is not expected to be manageable via the nova api
12:06:14 mohsen__ sean-k-mooney: Yes. My OpenStack environment already has a couple of instances up and running on it.
12:06:14 mohsen__ Thank you for your helpful response
12:13:16 sean-k-mooney darkhackernc: this looks like a masikari issue so i marked the bug as invliad for nova.
12:26:13 NicolaiRuckel jkulik: That really helped. I was able to fix the problem now.
12:31:37 jkulik NicolaiRuckel: nice!
12:36:21 opendevreview Lajos Katona proposed openstack/nova master: WIP Use SDK for Neutron https://review.opendev.org/c/openstack/nova/+/928022
13:09:25 mohsen__ sean-k-mooney: I migrated all the instances from one compute node to another computes. then reconfigured nova with the mentioned config above on the compute host where there aren't any instances no longer. then I ran "openstack resource provider inventory list <compute-host-uuid>" command and find out that the same issue still exists. what could be the root cause?
13:10:34 sean-k-mooney mohsen__: you have restared the compute agent on the host since it was drained?
13:11:49 mohsen__ sean-k-mooney: since I used kolla-ansible to reconfigure it, it has a handler which do the restart task after modification. I didn't do it manually but I can test it.
13:12:01 sean-k-mooney if so and the issue still happens i think your going to have to file a bug with some logs. i.e. the full error message or warning form the reouce tracker and or perodic task related to the createion or update of the resouce provider
13:12:44 sean-k-mooney mohsen__: you did the reconfiugre while there were isntances on the host correct
13:12:54 sean-k-mooney have you restarted it since it was empty
13:14:25 mohsen__ sean-k-mooney: I evacuated all the instances and then ran the kolla-ansible to reconfigure it. the restart task had been run after all the instances were evacuated.
13:18:04 mohsen__ sean-k-mooney: After evacuating all the instances and after reconfiguration, I manually restarted it and the issue still persist. sounds like there is no way unless reporting a bug🤔
13:18:15 sean-k-mooney ack, without inspecigng the logs and or error i cant really advise more. i likely wont have time to dig much deeper in the short term unfortunetly. i woudl suggeset filing a bug and trying to extract any relevent log messages (ideally at debug level) that are related to teh execution of the update_aviable_resouce provider perodic and the intiall startup of the agent related to
13:18:17 sean-k-mooney when it initally created teh RP invetories with the PCPUs inventory. you mentioned it initally was created and then lost correct? it would be good to try and capture the relevent logs showing both
13:21:16 mohsen__ sean-k-mooney: Thank you for your guidance
14:58:27 cardoe I know the nova meeting is coming up and I just want to ping on https://review.opendev.org/c/openstack/nova-specs/+/471815 I pinged earlier.
15:30:42 Uggla Nova meeting in ~30mn
16:01:39 Uggla #startmeeting nova
16:01:39 opendevmeet Meeting started Tue Aug 5 16:01:39 2025 UTC and is due to finish in 60 minutes. The chair is Uggla. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:39 opendevmeet Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:39 opendevmeet The meeting name has been set to 'nova'
16:01:51 Uggla Hello everyone
16:02:02 jssfr o/
16:02:04 jssfr NicolaiRuckel and me would like to bring up a topic in the "Open Discussion" slot
16:02:08 jssfr (jfyi)
16:02:29 fwiesel o/
16:02:44 Uggla jssfr, no pb but this week few people are available.
16:03:01 jssfr thanks for the heads up :)
16:03:08 gmaan o/
16:04:04 Uggla awaiting people to join, but I'm pretty sure we will not have quorum.
16:04:36 Uggla Please raise your hand for showing you are part of the meeting.
16:05:07 NicolaiRuckel o/
16:05:40 dansmith o/
16:06:03 Uggla Let's start
16:06:09 Uggla #topic Bugs (stuck/critical)
16:06:16 Uggla #info No Critical bug
16:06:58 melwitt o/
16:07:04 Uggla #topic Gate status
16:07:14 Uggla hey melwitt, good to see you
16:07:23 Uggla #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:07:29 Uggla #link https://etherpad.opendev.org/p/nova-ci-failures-minimal
16:07:36 Uggla #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status
16:07:48 Uggla #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:07:58 Uggla #info Please try to provide a meaningful comment when you recheck
16:08:06 Uggla #info Gate was blocked last week with "openstack.exceptions.SDKException: Image creation failed: 'latin-1' codec can't encode character '\u2019' in position 662: ordinal not in range(256) in openstack.tests.functional.image.v2.test_image.TestImage.test_tags".
16:08:13 Uggla #info https://review.opendev.org/c/openstack/openstacksdk/+/956369 fixed it
16:08:49 Uggla #topic tempest-with-latest-microversion job status
16:08:58 Uggla #link https://zuul.opendev.org/t/openstack/builds?job_name=tempest-with-latest-microversion&skip=0
16:09:10 Uggla gmaan, do you have something to share ?
16:10:28 gmaan ah, no update on this.
16:10:47 Uggla thanks, so moving on
16:10:55 Uggla #topic Release Planning
16:11:02 Uggla #link https://releases.openstack.org/flamingo/schedule.html
16:11:07 Uggla #info Nova deadlines are set in the above schedule
16:12:15 Uggla Time is flying, a bit less than 4 weeks before FF.
16:12:26 Uggla #topic Review priorities
16:12:36 Uggla #link https://etherpad.opendev.org/p/nova-2025.2-status
16:13:12 Uggla I guess the file is updated, please let me know if something is wrong in it.
16:13:29 Uggla #topic OpenAPI
16:13:37 Uggla #link: https://review.opendev.org/q/topic:%22openapi%22+(project:openstack/nova+OR+project:openstack/placement)+-status:merged+-status:abandoned
16:13:59 Uggla #info up from 13 to 32 remaining. (I guess Stephen created new ones and follow up)
16:14:24 Uggla I hope we can see an end about this.
16:15:01 Uggla I'm going to skip stable branch topic because Elod is on pto.
16:15:28 Uggla #topic vmwareapi 3rd-party CI efforts Highlights
16:15:36 Uggla fwiesel, something on our side ?
16:15:44 fwiesel #info fwiesel will be on PTO until the 9/9

Earlier   Later