Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2026-03-25
18:21:53 sean-k-mooney at the time this was approve the only usecase for a non defautl vnic type was neutron sriov prot which requried the port to be precreated.
18:21:59 cardoe Let's just abandon my proposed change. I'll go away.
18:22:20 sean-k-mooney cardoe: i think the idea of a hypervior depenent defualt vnic type is not a bad one
18:22:50 sean-k-mooney so im not against this just provideign the wider context
19:00:52 JayF Yeah I think we have the fix in line with cardoe's change, just need to figure out the right way to market it to at least one other Nova core and if needed update the paperwork to be in line with whatever is expected if they consider that a feature and not a bugfix
20:07:24 sean-k-mooney JayF: next step would be nova irc meeting. it would be better in the long term if we also had a path to growign more nova cores with ironic expertiese. in the past john garbut was one but they have less time for nova these days
20:07:39 sean-k-mooney JayF: im +1 on the patch over all.
20:08:39 JayF sean-k-mooney: similarly on the Ironic side; I've tried to take the mantle of "expert in the driver", which I think I've "hit" but many of the things here are lower level than that. Between the group of us we can find a solution I'm sure :)
20:10:58 opendevreview Antonin Ruan proposed openstack/nova-specs master: Temp URL download spec https://review.opendev.org/c/openstack/nova-specs/+/975883
20:22:00 opendevreview Erlon R. Cruz proposed openstack/nova master: Add RBD XML update functionality for migration process https://review.opendev.org/c/openstack/nova/+/974032
22:41:11 opendevreview Merged openstack/nova master: Update min support for Hibiscus https://review.opendev.org/c/openstack/nova/+/979301
#openstack-nova - 2026-03-26
08:47:09 priteau Hello Nova team. I have just hit the `mkfs.vfat: Label can be no longer than 11 characters` issue while live migrating an instance. There is a fix in Nova but backports are waiting for review. Could you please merge? https://review.opendev.org/c/openstack/nova/+/952003
10:04:22 priteau Uggla: Would it be possible to merge these backports?
10:09:05 Uggla priteau, I think so. I'll try to have them merged.
10:09:25 priteau Thank you!
10:16:16 gibi priteau: approved the backort now
10:36:15 opendevreview Doug Szumski proposed openstack/nova master: Stop corrupting ephemeral volumes during cold migration https://review.opendev.org/c/openstack/nova/+/940900
10:46:11 priteau Thanks gibi. I might ping you for the next ones!
10:47:06 gibi priteau: sure
10:58:38 nathharp hey knowledgable nova people. I'm having problems with driving Ironic via nova-compute: when building multiple instances, i see errors like: "status": 409, "title": "Conflict", "detail": "There was a conflict when trying to complete your request.\n\n resource provider generation conflict ", "code": "placement.concurrent_update". Is this likely to be an issue with nova, or placement? This is a 2025.1 cloud
11:23:00 nicolairuckel https://specs.openstack.org/openstack/nova-specs/specs/2025.1/approved/vtpm-live-migration.html Is it correct, that there is nothing to be implemented for the user mode since in that case live migration is not possible?
12:50:56 opendevreview Merged openstack/nova stable/2025.1: libvirt: Use common naming convention for ephemeral disk labels https://review.opendev.org/c/openstack/nova/+/952003
12:56:24 priteau gibi: 2025.1 backport has merged, thanks! 2024.2 backport is ready too: https://review.opendev.org/c/openstack/nova/+/952004
13:29:51 opendevreview Balazs Gibizer proposed openstack/nova master: Default native threading for conductor and compute https://review.opendev.org/c/openstack/nova/+/982237
14:32:20 opendevreview Kamil Sambor proposed openstack/nova master: Replace eventlet with standard threading in novncproxy https://review.opendev.org/c/openstack/nova/+/976089
15:09:45 opendevreview Merged openstack/nova stable/2024.2: libvirt: Use common naming convention for ephemeral disk labels https://review.opendev.org/c/openstack/nova/+/952004
16:06:11 opendevreview Kamil Sambor proposed openstack/nova master: Replace eventlet with standard threading in novncproxy https://review.opendev.org/c/openstack/nova/+/976089
17:11:13 clif sean-k-mooney: could you give me some context on ProviderTree? does the tree represent a heirarchical relationship between nodes? I also see ProviderTree calls out the ironic driver specifically for having 'large number of roots'
17:15:09 sean-k-mooney clif: placement is a forset data stucre where we have a forest of provider that are them selve aranged in a tree stucture with root RPs representign a comptue node and potically nested resouce provider as childer repcesting gpus or pci device on that compute host
17:15:50 sean-k-mooney clif: in the context of ironic each ironic node is create in placement as a seperate resouce provider with invetory of 1 resouce class fo custom_baremtal_something
17:16:18 sean-k-mooney so a single nova compute agent with the ironic driver manage a set of trees of depht 1
17:16:35 clif ah, ok
17:16:48 sean-k-mooney the uuid of the placment resouce provider is the ironic node uuid which is alsoce the nova compute node uuid
17:17:20 sean-k-mooney so it a 1:1:1 mapping between compute_node in the nova db, resouce provider in the placment db and node in the ironic db
17:17:52 clif so each tree is a single ironic node and each tree's children is information about the ironic node?
17:18:00 sean-k-mooney clif: for almost every other dirver each nova comptue agent manages exactly 1 compute node and 1 root resouce provider
17:18:35 sean-k-mooney clif: for hte ironic diver we dont create childer sinc you cant subdevide ironic hosts
17:18:39 JayF (this is, I presume, the root cause for how so much perf was left on the table in the Ironic case?)
17:18:45 sean-k-mooney we could in the future but we didnt have a suecase today
17:19:03 sean-k-mooney JayF: yes more or less
17:20:00 sean-k-mooney JayF: the first botlenec (the deep copy of the providers manged by this compute agent)
17:20:19 sean-k-mooney is cheap if the number of provider is 1-8 max in 99% of cases
17:20:42 sean-k-mooney its really expensive if you have 100s or 100s espcially if your doing it in a loop
17:21:05 clif could we create a seperate provider implementation for ironic driver use?
17:21:06 sean-k-mooney JayF: so i think that is 1 of the hopefully simple things to fix
17:21:31 sean-k-mooney clif: we try not to have any virt driver awareness in teh compute manager or resouce tracker
17:21:41 sean-k-mooney and that is where the provider stuff lives
17:22:01 sean-k-mooney but short answer is yes i think we can optimize it
17:22:06 clif alright
17:22:10 sean-k-mooney jsut without the need for an if ironic check
17:22:39 sean-k-mooney clif: that is what https://review.opendev.org/c/openstack/nova/+/980676 is ment to do
17:23:08 sean-k-mooney specifically https://review.opendev.org/c/openstack/nova/+/980676/1/nova/compute/resource_tracker.py#1314
17:23:27 clif right, I'm trying to load enough nova context into my noggin to get to the point where I can understand what's going on with that patch, the perf sim, etc
17:24:34 sean-k-mooney when we get the provider tree for a spcific node that shoudl copy only that node RP insated fo every provider in our provider cash https://review.opendev.org/c/openstack/nova/+/980676/1/nova/scheduler/client/report.py#935
17:25:29 clif it seems reasonable on the surface
17:25:35 sean-k-mooney clif: ya that a non zeor amount of context to load
17:26:07 sean-k-mooney the other optimisation that i think has prommis is this patch https://review.opendev.org/c/openstack/nova/+/980679/1
17:26:19 sean-k-mooney so that that one does is very simple
17:26:47 sean-k-mooney today we have a for loop over every ironic node managed by this agent (limited via shard key or peer_list)
17:27:06 sean-k-mooney the patch just kick the processing of each node out into a thread pool
17:27:19 sean-k-mooney so you can actully do them in parallel an tune it via a config option
17:27:28 sean-k-mooney defaulting to 1 for backward compatiablity
17:29:08 JayF And I hope this is all building up, once the identified low hanging fruit are picked, to a deeper look at if there's things we can do to improve it further. I am skeptical that this will be enough to bring us within an acceptable startup time :)
17:29:17 clif I will continue to delve
17:29:28 clif thanks for the context sean
17:30:17 sean-k-mooney JayF: these were the hotspots in the current profiling but if we can get a beter repoducer with the ironic fake driver or soemthign elese we can see what the next bottel neck is
17:30:53 JayF ironic devstack plugin support for fake driver nodes is now #2 on my todo list (I got the PRs up removing 15k worth of retired driver code lines from Ironic)
17:31:02 sean-k-mooney JayF: we could as you suggested skip the placement management entirly in nova and delegate that to ironic but i dont know if that will really help
17:31:26 sean-k-mooney we may need to consider other change liek batch updating placement which may need an api change
17:31:31 JayF sean-k-mooney: basically in the places we poll Ironic currently, any way to switch that to something more evented
17:31:37 JayF sean-k-mooney: or batching into placement, yep
17:31:58 sean-k-mooney well yes so we coudl extend the nova external event api for ironic
17:32:00 JayF sean-k-mooney: either way once we pluck the low hanging fruit I want us to take a look at doing the bigger stuff to help; myself and clif can be the hands making the action work
17:32:08 sean-k-mooney that woudl allow use to get ride of poolign for power state for example
17:32:11 JayF I just don't always know which way is up in Nova :)
17:32:54 sean-k-mooney JayF: if you are not familary with https://docs.openstack.org/api-ref/compute/#create-external-events-os-server-external-events
17:33:19 sean-k-mooney that is how cidner and nova tell nova that the sate of soemthign has changed or a workflow has completed in an event based manner
17:33:44 JayF Interesting
17:33:53 sean-k-mooney power-update might actuly be used by ironic already
17:34:06 sean-k-mooney but there is no reason that we coudl not add other event if we had a reason too
17:34:07 JayF I am 99.999999% it's not
17:34:46 sean-k-mooney im just tyring to think what ealse is power related that it could be
17:34:57 JayF yep we 100% do
17:35:00 sean-k-mooney https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id69
17:35:00 JayF TIL
17:35:27 JayF https://opendev.org/openstack/ironic/src/branch/master/ironic/common/nova.py#L91
17:35:51 sean-k-mooney yep so i think this was added so we can turn off the power monitring perodic in ironic deployments
17:35:52 JayF if we could just trigger update of placement relevant info from Ironic e.g. on certain state changes
17:36:12 JayF I don't care if any of that comm goes Ironic<>Nova<>Placement or Ironic<>Placement
17:36:30 sean-k-mooney well ironic can go do placmeent adn do some limited things today
17:36:43 JayF especially if we were able to somehow get those calls to impact the node cache inside the virt driver itself
17:37:01 sean-k-mooney JayF: so if you tell me waht you want to update there i can say if it doable today or not without nova changes
17:37:05 JayF because then we can do the "reconciliation job on a timer" change like you did that I already -1'd
17:37:50 JayF sean-k-mooney: I don't know that answer (yet); but whatever happens when a node goes unavailable (e.g. cleaning or move out of AVAILABLE state) and whatever happens when a node goes available (anything->AVAILABLE state)
17:38:16 JayF Plus maybe something that would trigger on a node trait add to expose that trait to placement
17:38:29 sean-k-mooney so the way to model that in placement today is to set reserved=total on the rp
17:38:49 sean-k-mooney that is something that ironic can do directly
17:38:58 JayF Right now my goal is to get all these written down so we can evaluate them after the lower hanging fruit have been plucked
17:39:09 JayF because hopefully by then we'll have devstack w/fake driver nodes
17:39:10 sean-k-mooney ack sound good

Earlier   Later