| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-12-09 | |||
| 12:06:59 | noonedeadpunk | let me fix scheduler anyway... | |
| 12:07:10 | sean-k-mooney | noonedeadpunk: can you check if the resouce provider for that compute exists in placement | |
| 12:07:31 | noonedeadpunk | so the id of in the stack trace is a child | |
| 12:07:39 | noonedeadpunk | https://paste.openstack.org/show/bn6ekRrWbg9ggNpuGx4h/ | |
| 12:07:47 | sean-k-mooney | ack ya it shoudl be the chile for the pci device | |
| 12:08:04 | sean-k-mooney | right so its os-compute-gpu01-az2_0000:25:00.0 | |
| 12:08:12 | noonedeadpunk | yeah | |
| 12:08:16 | sean-k-mooney | the inventory for the gpu | |
| 12:08:37 | sean-k-mooney | so can you confirm that that is in you pci devspec and that the adress is correct | |
| 12:09:11 | noonedeadpunk | 25:00.0 3D controller: NVIDIA Corporation | |
| 12:09:14 | noonedeadpunk | from lspci | |
| 12:09:41 | sean-k-mooney | ya that looks normal | |
| 12:10:28 | sean-k-mooney | i belive self._remove_managed_rps_from_tree_not_in_view(provider_tree) is intended to be used to remove RP for pci device that are nolonger in the whitelist | |
| 12:10:48 | sean-k-mooney | so that is why i was suggestign checkign the devspec in the nova.conf on the compute to confirm that is correct | |
| 12:12:34 | sean-k-mooney | there could ligitimatly be a bug i just want to rule out config errors | |
| 12:13:15 | noonedeadpunk | so one thing I am suspicious about.... | |
| 12:13:47 | noonedeadpunk | Is that we have a custom trait on top of the resource class in there | |
| 12:14:00 | noonedeadpunk | and this custom trait is applied only to the root device | |
| 12:14:20 | noonedeadpunk | https://paste.openstack.org/show/bhtdPwgzYHpzCdoIGe32/ | |
| 12:14:24 | sean-k-mooney | am i dont think that shoudl break anythign but maybe | |
| 12:15:26 | noonedeadpunk | ah, well, it's passed to children as well | |
| 12:15:33 | sean-k-mooney | we shoudl be adding COMPUTE_MANAGED_PCI_DEVICE as well | |
| 12:15:39 | sean-k-mooney | can you check fi you see both? | |
| 12:16:38 | noonedeadpunk | yes, I see CUSTOM_GPU_PASSTHROUGH and COMPUTE_MANAGED_PCI_DEVICE traits on both parent and childs | |
| 12:16:48 | noonedeadpunk | but not the resource_class | |
| 12:17:05 | sean-k-mooney | well the resouce class shoul be the resouce class in the invetnotry | |
| 12:17:11 | noonedeadpunk | But resource class should not be in traits iirc | |
| 12:17:18 | sean-k-mooney | correct | |
| 12:17:27 | sean-k-mooney | if you do an inventory show on the rp you shoudl see it there | |
| 12:17:42 | noonedeadpunk | openstack resource class show CUSTOM_A10_FULL returns the name, yes | |
| 12:20:57 | noonedeadpunk | I kinda failing to understand at what point `self.roots_by_uuid` is expected to be populated | |
| 12:22:16 | sean-k-mooney | https://opendev.org/openstack/nova/src/commit/8a4b000216c7a6c2673af78d7eb7f9bf938dc867/nova/compute/provider_tree.py#L292 | |
| 12:22:17 | noonedeadpunk | it does not look like it's cached, or populated during instance create request | |
| 12:22:47 | noonedeadpunk | but according to trace it's not called? | |
| 12:22:50 | sean-k-mooney | it can be populated in othere wase too | |
| 12:23:10 | sean-k-mooney | the tace doe not actully show a specific error | |
| 12:23:20 | sean-k-mooney | just that the child rp was nto found | |
| 12:23:43 | sean-k-mooney | i assume this si repoabale i.e. if yuou restart the nova comptue agent it happens again | |
| 12:24:02 | noonedeadpunk | well, I used epdb, and self.roots is dict_values([]) | |
| 12:24:10 | noonedeadpunk | yes, sure | |
| 12:24:43 | sean-k-mooney | its returnnign the values mapping of the dict yes | |
| 12:25:17 | noonedeadpunk | and self.roots_by_uuid is {} | |
| 12:25:27 | sean-k-mooney | this shoudl be updated in memory as part fo init_host | |
| 12:25:31 | noonedeadpunk | so it feels that it's not populated in fact... | |
| 12:26:23 | sean-k-mooney | so during init_host we end up running update_aviable_resocues which populates the resouce tacker and caches the provieder tree | |
| 12:27:28 | sean-k-mooney | this should be poplulated before we ever get to _build_and_run_instance | |
| 12:29:36 | noonedeadpunk | unless class is not re-inited at some point.... | |
| 12:36:28 | opendevreview | Dominik proposed openstack/nova master: Regression test for Placement allocations remaining during failed schedule https://review.opendev.org/c/openstack/nova/+/969251 | |
| 12:36:50 | opendevreview | Dominik proposed openstack/nova master: Remove Placement allocations in the broken build cleanup https://review.opendev.org/c/openstack/nova/+/968446 | |
| 12:38:36 | DominikDanelski[m] | sean-k-mooney: All right, I added a comment explaining why the exact flavour doesn't matter, so now all discussion is resolved. | |
| 12:39:33 | noonedeadpunk | sean-k-mooney: so I reverted 78be1679312768383b684fa70ca4d2f5c4e35fa9 and instance become spawning normally :( | |
| 12:40:16 | noonedeadpunk | will submit a bug report in the meanwhile I guess | |
| 12:40:38 | sean-k-mooney | an do they have allcoation against the placemetn RPs for the gpu usage? | |
| 12:41:00 | sean-k-mooney | i..e there were no other sideffects? | |
| 12:41:21 | sean-k-mooney | im wondering if we are misisng a fix for one of the ohter caching bugs related to OTU devices | |
| 12:41:28 | sean-k-mooney | in the sable branch | |
| 12:41:45 | sean-k-mooney | gibi and dan were fixing some other bugs around the same time | |
| 12:44:10 | noonedeadpunk | it looks fine? https://paste.openstack.org/show/bj8ZZgxzHSYHZF2hGFvg/ | |
| 12:45:00 | noonedeadpunk | hm, maybe I am missing some backport... | |
| 12:45:47 | noonedeadpunk | nah, I think we're on current HEAD of 2025.1 | |
| 12:47:37 | sean-k-mooney | ya that looks correct to me too | |
| 13:00:02 | noonedeadpunk | created report https://bugs.launchpad.net/nova/+bug/2134469 | |
| 13:21:04 | DominikDanelski[m] | sean-k-mooney: Sorry, then do I understand correctly what you wrote at 12:55 that if I replied to the comment but made no code changes, just as I did now, you wouldn't be informed about it? | |
| 13:31:35 | DominikDanelski[m] | I wouldn't want to constantly bug you here, but then I don't know to best reach you when I reply. | |
| 13:50:47 | sean-k-mooney | DominikDanelski[m]: so every unless you adjust it in you settings if your on a review (reviewwr or cc) you will get an email every time somone comments or pushes a new patch | |
| 13:51:32 | sean-k-mooney | DominikDanelski[m]: and it show up with your name rater then zuul in the from | |
| 13:51:42 | sean-k-mooney | i.e. Dominik (Code Review) | |
| 13:51:58 | sean-k-mooney | so i will be notifed by ether a comment or a new aptchset | |
| 13:52:18 | sean-k-mooney | i just might not see it for a while depending on how often i check my email | |
| 15:26:04 | nicolairuckel | I'm a bit lost trying to figure out which code gets called for a cold migration. There is the `_cold_migrate` function in manager.py but I kind of expected a call to the driver at some point where I could copy the NVRAM but I can't find anything like that. Am I overlooking something or am I on the wrong track to begin with? | |
| 15:48:14 | tkajinam | nicolairuckel, check migrate_disk_and_power_off in LibvirtDriver | |
| 15:48:30 | tkajinam | I think nova internally calls resize method | |
| 17:15:01 | nicolairuckel | tkajinam: Hm, that's what I tried first but it didn't work. I'm going to check it again. Maybe I did something wrong when I was testing it. Thank you. | |
| 18:03:12 | opendevreview | Nicolai Ruckel proposed openstack/nova master: Preserve UEFI NVRAM variable store https://review.opendev.org/c/openstack/nova/+/959682 | |
| 18:04:31 | nicolairuckel | I pushed my attempt so maybe someone else sees anything that is obviously wrong there. | |
| 18:56:50 | nicolairuckel | This is what I tried: https://review.opendev.org/c/openstack/nova/+/959682/comment/2c08400d_baac062c/ | |
| 19:31:32 | opendevreview | Zhan Zhang proposed openstack/nova-specs master: Refine network setup procedure in live migrations https://review.opendev.org/c/openstack/nova-specs/+/970298 | |
| 19:35:52 | Zhan[m] | sean-k-mooney: As we previously discussed I've submitted the spec for https://bugs.launchpad.net/nova/+bug/2128665, unfortunately didn't make it for 2026.1 so this will be in 2026.2. | |
| 19:42:49 | opendevreview | Zhan Zhang proposed openstack/nova-specs master: Refine network setup procedure in live migrations https://review.opendev.org/c/openstack/nova-specs/+/970298 | |
| 19:54:16 | opendevreview | Zhan Zhang proposed openstack/nova-specs master: Refine network setup procedure in live migrations https://review.opendev.org/c/openstack/nova-specs/+/970298 | |
| #openstack-nova - 2025-12-10 | |||
| 03:07:46 | opendevreview | yangjianfeng proposed openstack/nova master: Fix missing allocations due to unsuccessful evacuations https://review.opendev.org/c/openstack/nova/+/970335 | |
| 05:10:42 | opendevreview | ChungWon Lee proposed openstack/nova master: Fix error when multiple security group in a project with same name https://review.opendev.org/c/openstack/nova/+/970337 | |
| 05:15:20 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add functional reproducer for bug 2096884 https://review.opendev.org/c/openstack/nova/+/970238 | |
| 05:46:40 | opendevreview | ChungWon Lee proposed openstack/nova master: Fix error when multiple security group in a project with same name https://review.opendev.org/c/openstack/nova/+/970337 | |
| 08:02:59 | cw0306-lee[m] | Hi, nova team!... (full message at <https://matrix.org/oftc/media/v1/media/download/ATEBK2ecPfYUpbLTNfvf7ZZI5ImOneLHiJwCBCzpzpfS8rYqhQHpXqp3Yg5cVsWTSml0Rob0cV8Er3o0OIgJL7RCebT2H1TAAG1hdHJpeC5vcmcvVEJRdG5zdmxzcHBvRElxTWhxYndZdFRT>) | |
| 09:06:43 | opendevreview | Rajesh Tailor proposed openstack/nova master: Clean up volume attachments when server create fails https://review.opendev.org/c/openstack/nova/+/970350 | |
| 10:00:51 | opendevreview | Rajesh Tailor proposed openstack/nova master: Clean up volume attachments when server create fails https://review.opendev.org/c/openstack/nova/+/970350 | |
| 13:16:20 | opendevreview | Rajesh Tailor proposed openstack/nova master: Clean up volume attachments when server create fails https://review.opendev.org/c/openstack/nova/+/970350 | |
| 14:33:39 | sean-k-mooney | ratailor: melwitt ^ https://review.opendev.org/c/openstack/nova/+/970350 looks like a a duplciate or related to work started by DominikDanelski[m] on https://bugs.launchpad.net/nova/+bug/2132020 in https://review.opendev.org/c/openstack/nova/+/969251/6 and https://review.opendev.org/c/openstack/nova/+/968446/3 | |
| 14:35:48 | sean-k-mooney | ratailor: melwitt the quta exceed part of https://bugs.launchpad.net/nova/+bug/2096884 is likely a sideeffect of the fact we do nto delete the allcoation when the server errors out as reported in https://bugs.launchpad.net/nova/+bug/2132020 | |
| 17:14:12 | JayF | https://review.opendev.org/c/openstack/nova/+/969833 is proposed for stable/2024.2, +2'd already by Elod, V+1, ready to merge. Please help me land it? | |
| 17:17:39 | sean-k-mooney | sure | |
| 17:18:37 | sean-k-mooney | JayF: done | |
| 17:18:46 | JayF | thank you | |
| 17:18:57 | JayF | now to wonder into the unmaintained wastes to land it further :P | |
| 17:20:00 | sean-k-mooney | i intentially do not have merge rights there so this is as far as i cna help but i think elod still does and we dont enforce mutliple +2 there so once this is merge they can probly hlep with older branches | |
| 17:20:13 | JayF | I am unmaintained core as well :D | |
| 17:20:24 | JayF | I put the link (https://review.opendev.org/c/openstack/nova/+/970448) for 2024.1 in #openstack-unmaintained | |
| 18:10:41 | noonedeadpunk | sooo... about https://review.opendev.org/c/openstack/nova-specs/+/900296/13/specs/2026.1/approved/cross-az-instance-scheduling.rst | |
| 18:11:01 | noonedeadpunk | just to be clear - should I be adding 409 to catch AZ conflict or not? | |