| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-08-15 | |||
| 10:20:35 | sean-k-mooney | is there a reason you did that | |
| 10:20:54 | sean-k-mooney | im not saying we should not look into the clean up | |
| 10:21:12 | zigo | It's just our procedure to clean-up nova, cinder and neutron services when we decomission a node... | |
| 10:21:17 | sean-k-mooney | but the procudure yoru takign is not what i would expect/recommend if you are replacing a node bug keeping the host name | |
| 10:21:30 | zigo | Ah. | |
| 10:21:56 | sean-k-mooney | zigo this is what i woudl expect if you were chanign the hostname i.e scale in and out | |
| 10:22:12 | sean-k-mooney | but not if you are preserving the host name | |
| 10:23:22 | sean-k-mooney | i think that might be why we dont see this downstream | |
| 10:24:16 | sean-k-mooney | we have 2 diffent but related procedures. if its just a hardware failure and you want to just swap out the failed hardware then we woudl expct you do phsyicall do that and then just reinstalll but not delete the compute service | |
| 10:24:50 | sean-k-mooney | for hosts where the instances are on cpeh for example that allows replacement of the failed hardware even if you have not evacuated the instances although that is prefered | |
| 10:25:25 | sean-k-mooney | in our old instally scalining in and out a new server ended up with a new hostname | |
| 10:25:35 | sean-k-mooney | so it would not hit this edgecases | |
| 10:25:58 | zigo | sean-k-mooney: I would expect that I can do what I want, the way I want, without experiencing such a bug ... :) | |
| 10:29:50 | sean-k-mooney | well you can expect that :) | |
| 11:21:30 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926407 | |
| 13:40:41 | sean-k-mooney | gibi: i think what you are proposing is a better fix then mine as you have moved the cleanup to the correct function | |
| 13:40:57 | sean-k-mooney | gibi: but i left a comment inline. | |
| 13:42:47 | sean-k-mooney | im not sure we will have time to sync with artom beofre our internal meeting start so we might just do that after we finish wiht them or we can do it async | |
| 13:43:16 | sean-k-mooney | and check back tomorow | |
| 13:53:39 | gibi | let's try tomorrow. I don't think I will have the mental capacity after our internal call. But I will clean up the unit and functional test failures at least in the meantime | |
| 13:54:01 | gibi | also I agree about free_instance_claims but I will check for fallouts | |
| 13:54:34 | artom | sean-k-mooney, gibi, what's up? | |
| 13:54:57 | sean-k-mooney | artom: gibi foudn the root cause of the bug | |
| 13:55:19 | artom | Sweet | |
| 13:55:48 | gibi | artom: this is the distilled version https://review.opendev.org/c/openstack/nova/+/926407 | |
| 13:56:12 | gibi | the longer form is in the comments of https://review.opendev.org/c/openstack/nova/+/710848/13#message-354946be57b39eb88b67872230c226361d8a3559 | |
| 13:57:09 | artom | Oh wow | |
| 13:57:11 | gibi | I have still 30 mins befor the next call so if you artom, sean-k-mooney want to sync on it then I can explain in gmeet | |
| 13:57:11 | sean-k-mooney | artom: i was suggeting this morning that we coudl hop on a call and try and find it together but we said we woudl wait for you to come online before doing that and in the mean time gibi made some progress | |
| 13:57:34 | artom | That is so insidious, well found! | |
| 13:58:09 | artom | Of course then the fear is - what else are we affecting, and potentially breaking? | |
| 13:58:15 | sean-k-mooney | im ment ot have a 1:1 in 3 minute but not sure if its going to happen | |
| 13:58:21 | sean-k-mooney | artom: right | |
| 13:58:31 | artom | I feel like our test coverage is enough at this point that's it's _probably_ fine | |
| 13:58:47 | sean-k-mooney | maybe | |
| 13:59:04 | gibi | artom: exactly. I also a bit affraid of the potential fallout | |
| 13:59:20 | gibi | the functional test is clean (exept one test case that I can explain and fix) | |
| 13:59:22 | sean-k-mooney | so what i was hoping is you could integrate the changes form gibi and try an fix the test coverage as needed | |
| 13:59:38 | sean-k-mooney | then we could see what that looks liek tomorrow | |
| 14:00:07 | gibi | sean-k-mooney: artom: I can clean up my patch so artom can focus on the socket id thing | |
| 14:00:17 | sean-k-mooney | ack that works too | |
| 14:00:24 | gibi | and we can ask melwitt or dansmith for review as a second core | |
| 14:01:20 | sean-k-mooney | speaking of mel i need to loop back to here unified limits changes | |
| 14:01:33 | sean-k-mooney | melwitt: did the test change to oslo.limits land yet | |
| 14:02:21 | sean-k-mooney | melwitt: oh ok you abandoned https://review.opendev.org/c/openstack/oslo.limit/+/924024 | |
| 14:02:49 | sean-k-mooney | so i guess the intent is to proceed without that for now in https://review.opendev.org/c/openstack/nova/+/924025 | |
| 14:04:03 | sean-k-mooney | melwitt: i have not looked at that in a while but i see you still have it marked as work in progress in gerrit | |
| 14:04:32 | sean-k-mooney | are you happy the current version is ready for review or should i loop back in a few days | |
| 14:50:17 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926407 | |
| 15:00:16 | melwitt | sean-k-mooney: yeah the intent was to go for a nova-only approach. and oops, not supposed to be marked as WIP anymore, I'll unmark them | |
| #openstack-nova - 2024-08-16 | |||
| 03:27:30 | opendevreview | melanie witt proposed openstack/nova master: Add [workarounds]unified_limits_permissive_enforcement https://review.opendev.org/c/openstack/nova/+/924025 | |
| 06:53:38 | opendevreview | melanie witt proposed openstack/nova master: Add [workarounds]unified_limits_permissive_enforcement https://review.opendev.org/c/openstack/nova/+/924025 | |
| 08:25:39 | gibi | artom: sean-k-mooney: bauzas: you probably remember when we looked at why unshelving to an ironic host fails with the cryptic error as Destination.node is not set while unshelving to a libvirt host works even if the compute api only sets Destination.host there too. The answer is in https://bugs.launchpad.net/nova/+bug/2075504/comments/3 | |
| 08:26:04 | gibi | basically the scheduler fills Destination.node based on .host if the mapping is 1:1 | |
| 08:45:28 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926407 | |
| 10:00:47 | sean-k-mooney[m] | i didnt think shelve was supported for ironic | |
| 10:01:28 | sean-k-mooney[m] | i know unselve could work but i would expect that to not be supproted as it would require chaning the flavor | |
| 10:01:51 | sean-k-mooney[m] | since ironic nodes should bnot have cpu, ram or disk | |
| 10:05:15 | sean-k-mooney[m] | and should be using a flavor with only a custom resource class | |
| 10:05:15 | sean-k-mooney[m] | so placement should retrun no allocation for the host | |
| 10:09:30 | gibi | sean-k-mooney: yes, it is not supported but it fails in a very stange way | |
| 10:09:44 | gibi | so I have that cosmetic bug open to fail it in a more clear way | |
| 10:10:20 | gibi | and while we discussed that we had an open question how don't unshelve to libvirt node does not fail with the same reason, the missing Destination.node field | |
| 10:10:25 | sean-k-mooney[m] | there is also a tempet/job config bug. there is a temest setting for pining which host is used for the source and dest node for migrations | |
| 10:10:26 | gibi | and yesterday I found the reason | |
| 10:10:48 | sean-k-mooney[m] | we either are not setting it and shelve tests dont respect it or we need to add that to the job config | |
| 10:10:54 | gibi | sean-k-mooney: yeah, also James is fixing the host selection logic for unshelve to host | |
| 10:11:00 | gibi | in tempest | |
| 10:11:00 | sean-k-mooney[m] | that or we need to add a shelve sepcific config for that | |
| 10:11:09 | sean-k-mooney[m] | ok | |
| 10:11:50 | sean-k-mooney[m] | we should proably return an exception in the 1:many case | |
| 10:12:35 | gibi | yeah that would probably make sense otherwise the scheduler blows up later anyway | |
| 10:13:03 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926407 | |
| 12:54:39 | opendevreview | Takashi Natsume proposed openstack/nova-specs master: Create specs directory for 2025.1 Epoxy https://review.opendev.org/c/openstack/nova-specs/+/924227 | |
| 13:31:19 | opendevreview | sean mooney proposed openstack/nova master: split monkey_patching form import https://review.opendev.org/c/openstack/nova/+/922425 | |
| 13:31:20 | opendevreview | sean mooney proposed openstack/nova master: refactor cmd dir for non eventlet console_scripts https://review.opendev.org/c/openstack/nova/+/904424 | |
| 13:31:20 | opendevreview | sean mooney proposed openstack/nova master: Remove nova debugger funcitonality https://review.opendev.org/c/openstack/nova/+/922496 | |
| 13:31:21 | opendevreview | sean mooney proposed openstack/nova master: Add IO Thread Pool Executor https://review.opendev.org/c/openstack/nova/+/922497 | |
| 13:31:21 | opendevreview | sean mooney proposed openstack/nova master: remove eventlet.tpool https://review.opendev.org/c/openstack/nova/+/905287 | |
| 13:31:22 | opendevreview | sean mooney proposed openstack/nova master: use futureist for scatter_gather https://review.opendev.org/c/openstack/nova/+/905284 | |
| 13:31:22 | opendevreview | sean mooney proposed openstack/nova master: use futerist for RBD calls https://review.opendev.org/c/openstack/nova/+/917962 | |
| 13:31:57 | sean-k-mooney | stephenfin: gibi: ^ rebased and adress the current out standing issues if you have time to review | |
| 13:32:09 | stephenfin | aclk | |
| 13:55:26 | opendevreview | sean mooney proposed openstack/nova master: add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | |
| 13:55:27 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck manager to manager base https://review.opendev.org/c/openstack/nova/+/827844 | |
| 13:55:27 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck tracker to nova context https://review.opendev.org/c/openstack/nova/+/829468 | |
| 13:55:28 | opendevreview | sean mooney proposed openstack/nova master: add healthcheck endpoint to proxy commands https://review.opendev.org/c/openstack/nova/+/830703 | |
| 13:55:28 | opendevreview | sean mooney proposed openstack/nova master: [WIP] add iniital health check to compute agent https://review.opendev.org/c/openstack/nova/+/906911 | |
| 13:55:29 | opendevreview | sean mooney proposed openstack/nova master: [WIP] add libvirt connection healtcheck https://review.opendev.org/c/openstack/nova/+/907424 | |
| 13:55:47 | sean-k-mooney | thats ^ just a rebase i need to go though them and see what review feedback is present | |
| 14:10:29 | opendevreview | Balazs Gibizer proposed openstack/nova master: Fix PCI passthrough cleanup on reschedule https://review.opendev.org/c/openstack/nova/+/926407 | |
| 14:34:11 | opendevreview | Zhang Hua proposed openstack/nova master: BlockDeviceMapping object has no attribute copy https://review.opendev.org/c/openstack/nova/+/920374 | |
| 15:07:06 | opendevreview | Arnaud Morin proposed openstack/nova master: libvirt: Log exception if failure checking block job https://review.opendev.org/c/openstack/nova/+/907720 | |
| 16:27:00 | gibi | artom: sean-k-mooney: the running CI looking green on https://review.opendev.org/c/openstack/nova/+/926407 and I fixed up all the comments. | |
| 16:27:24 | gibi | that is my last contribution for this week and next week I'm off. | |
| 16:27:43 | gibi | see you on the 26th | |
| 17:19:25 | sean-k-mooney | gibi: thanks gibi im pretty happy with that patch | |
| 17:19:36 | sean-k-mooney | ill see if i can chase other to review while your on pto, enjoy | |
| 22:03:32 | opendevreview | Merged openstack/nova master: api: Add response body validation helper https://review.opendev.org/c/openstack/nova/+/915741 | |
| 22:35:47 | opendevreview | Merged openstack/nova master: hardware: Correct log https://review.opendev.org/c/openstack/nova/+/926223 | |