| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-04-03 | |||
| 15:12:22 | dansmith | "enough bugs" makes it controversial? | |
| 15:12:40 | sean-k-mooney | sorry not controversial | |
| 15:13:00 | dansmith | ah | |
| 15:13:49 | sean-k-mooney | the approch to fixign it might be, i dont know but the fact the current bavhior is surpising to poeple is demonstarted by the exising bug reports | |
| 15:14:23 | dansmith | right, I meant the actual approach might be controversial | |
| 15:14:35 | dansmith | but I'd definitely like the opportunity to make my case for why I think this approach is the right one | |
| 15:14:56 | sean-k-mooney | i dont really see many other options to fixing it that would be backportable so that a vote in favor of the appoch | |
| 15:15:07 | sean-k-mooney | but ya lets see what others think next week | |
| 15:15:38 | dansmith | yeah, I agree of course.. but people may have religious/idealistic objections, idk | |
| 15:15:52 | sean-k-mooney | as an aside i might call it soon. im feellign progressively worse. soim going to take some panadol and get a hot drink if that does nto help ill call it a day | |
| 15:16:28 | dansmith | ack, melwitt just approved the traits patch | |
| 15:18:00 | opendevreview | Merged openstack/os-traits master: Add traits for sound and USB devices on instances. https://review.opendev.org/c/openstack/os-traits/+/940418 | |
| 15:24:51 | sean-k-mooney | cool melwitt ++ | |
| 15:31:31 | melwitt | sean-k-mooney: just in case you missed it, from your comment I added a repro func test under this and also a reno https://review.opendev.org/c/openstack/nova/+/880399 | |
| 15:32:36 | melwitt | note that the func test doesn't cover the "old" cinder API path, I wasn't sure if it needs to i.e. I wasn't sure if the old Cinder path is something we expect to be possible at this point/release | |
| 15:34:59 | sean-k-mooney | i had not seen that | |
| 15:35:03 | sean-k-mooney | but cool | |
| 15:35:45 | sean-k-mooney | on a related not i have a cinder volume multi attch pathc i need to clean up and push again.. | |
| 15:37:12 | sean-k-mooney | https://review.opendev.org/q/topic:%22bug/2048837%22 but its in merge conflict so no need ot look now. | |
| 15:37:38 | sean-k-mooney | there are a couple of volume related pathces in diffetn state aroudn at the moment | |
| 15:39:39 | melwitt | ack | |
| 15:40:26 | sean-k-mooney | weried os before we alwasy faked the volujme was attached even fi it was not assocated ot teh vm | |
| 15:41:51 | melwitt | what? :) | |
| 16:14:33 | sean-k-mooney | melwitt: https://review.opendev.org/c/openstack/nova/+/946222/1/nova/tests/fixtures/cinder.py#271 | |
| 16:14:42 | sean-k-mooney | i was commenting on the addtion of the if | |
| 16:16:17 | sean-k-mooney | not that what i orginally type convayed that particallary well | |
| 16:26:44 | melwitt | sean-k-mooney: oh haha | |
| 16:27:37 | melwitt | yeah the cinder fixture is a little wonky | |
| 16:47:34 | sean-k-mooney | +1 but im not entirlly sure how stable the looping is | |
| 16:47:41 | sean-k-mooney | comments inlien | |
| 16:48:05 | sean-k-mooney | we often use threading.evetns of locs when we want to encfoce a specific ordering | |
| 16:49:18 | sean-k-mooney | melwitt: im going to run your repoducer in a loop for a bit and see if its sable | |
| 16:49:24 | sean-k-mooney | melwitt: have you done that? | |
| 16:49:37 | melwitt | sean-k-mooney: no I didn't | |
| 16:50:50 | melwitt | I'll adjust things based on your comments. I haven't really done a func test like this before where it is more about checking the state of a resource in a different component, so I may have done it not the best way | |
| 16:51:36 | sean-k-mooney | well i think yoru test is fine. i just know that we have had issues with retry loops in fucntional tests before | |
| 16:51:49 | sean-k-mooney | so if we can force the ordering more explcitly that can help | |
| 16:52:10 | sean-k-mooney | ill let this run for a while and if it passes 50-100 times its proably ok | |
| 16:52:49 | sean-k-mooney | i think we can impove things if we reuse some of the helpers but im no tsure we strictly need too for this to prove what it needs too | |
| 16:53:36 | sean-k-mooney | ya the cidner fixutre behvaior is a litttel odd in places | |
| 16:53:50 | sean-k-mooney | but its also non trivial to emulate os im not really surpised | |
| 16:54:24 | sean-k-mooney | melwitt: ah https://review.opendev.org/c/openstack/nova/+/919979/4/nova/tests/functional/integrated_helpers.py | |
| 16:54:56 | sean-k-mooney | i _attch_volume for my multi attach repoducer | |
| 16:55:01 | melwitt | yeah I think my concern with it is if it could possibly "skip" the status "detaching" due to timing differences. so I won't be surprised if need to do the wait a different way | |
| 16:55:12 | sean-k-mooney | i was very confused why i did nto find it when i went looking | |
| 16:56:29 | sean-k-mooney | melwitt: so there are some exmples form gibi useing treading.event | |
| 16:56:36 | sean-k-mooney | to get thisn to pause in a specific state | |
| 16:56:41 | melwitt | oh my bad, I either didn't see that or there was some reason it didn't work for me and I forgot. but more likely the former | |
| 16:56:42 | sean-k-mooney | mainly for networking i think | |
| 16:57:01 | sean-k-mooney | no _attach_volume is not merged | |
| 16:57:12 | sean-k-mooney | its in that other volume serises i mentioend to you that i need to respine | |
| 16:57:38 | melwitt | when I first started this test I used libvirt func test base which super did not work and is not needed anyway and I changed it | |
| 16:57:40 | melwitt | oh haha ok, didn't realize that was an unmerged patch | |
| 16:58:07 | sean-k-mooney | so i last worked ont hat patch 6 months ago so in my brain its totally merged | |
| 16:58:31 | melwitt | :) | |
| 16:58:40 | sean-k-mooney | but i keep not geting around to rebaseing it | |
| 17:00:13 | sean-k-mooney | melwitt: so this is what i was referint too https://github.com/openstack/nova/blob/master/nova/tests/functional/regressions/test_bug_2085975.py | |
| 17:00:30 | sean-k-mooney | using trheading event to allow use to pause thing in speciifc places | |
| 17:00:42 | melwitt | ah ok, thanks | |
| 17:01:43 | sean-k-mooney | that might not be appliable to this case but its ther if you thing it would be useful basicly you woudl set the evnet in the palce your raising | |
| 17:02:09 | sean-k-mooney | and we woudl wait on it were we currently have the retry loop | |
| 17:02:50 | melwitt | gotcha. yeah that might be nicer | |
| 17:03:15 | sean-k-mooney | ====== | |
| 17:03:17 | sean-k-mooney | Totals | |
| 17:03:19 | sean-k-mooney | ====== | |
| 17:03:21 | sean-k-mooney | Ran: 100 tests in 700.9413 sec. | |
| 17:03:23 | sean-k-mooney | - Passed: 100 | |
| 17:03:25 | sean-k-mooney | - Skipped: 0 | |
| 17:03:27 | sean-k-mooney | - Expected Fail: 0 | |
| 17:03:29 | sean-k-mooney | - Unexpected Success: 0 | |
| 17:03:31 | sean-k-mooney | - Failed: 0 | |
| 17:03:33 | sean-k-mooney | Sum of execute time for each test: 353.3871 sec. | |
| 17:03:35 | sean-k-mooney | i mean it seam to work as it is | |
| 17:03:37 | sean-k-mooney | so im less concerned now | |
| 17:05:54 | melwitt | yeah I think what could mess it up is if the timing was sufficiently bad, like what if the _wait got delayed due to CI node busyness and it misses the intermediate state of "detaching". so I think you're right there is some instability possible there | |
| 17:06:26 | sean-k-mooney | ya it could happen. we could rehceck it once or twice | |
| 17:06:42 | sean-k-mooney | its a test only change so it does not run tempest | |
| 17:06:47 | sean-k-mooney | so rechecking is cheap | |
| 17:06:52 | melwitt | yeah. I'll try the event thing. oh right | |
| 17:07:43 | sean-k-mooney | ack, no worreis if you dont hcange it. i upgraded to +2 and rechecked | |
| 17:07:50 | melwitt | and also I was thinking future failures after it merged | |
| 17:07:51 | sean-k-mooney | if we recheck once more after that | |
| 17:07:58 | sean-k-mooney | i think we are good | |
| 17:08:15 | sean-k-mooney | ya its the future case that i was thinking of | |
| 17:08:25 | sean-k-mooney | we have loops like this today | |
| 17:08:30 | sean-k-mooney | but we have had issues with them too | |
| 17:08:45 | sean-k-mooney | so im alwasy cautious when adding more | |
| 17:08:55 | melwitt | yeah. also probably wastes some time due to the loop | |
| 17:09:52 | sean-k-mooney | technially yes but its 3.5 seccond pretty reliably on my laptop | |
| 17:09:58 | sean-k-mooney | not amazing but not terrible | |
| #openstack-nova - 2025-04-04 | |||
| 07:24:42 | zigo | FYI, we were affected by nova-compute flapping, we applied https://review.opendev.org/c/openstack/nova/+/939317 and that solved the issue for us. Thanks to melwitt and others who worked on it. | |
| 08:06:39 | opendevreview | Masahito Muroi proposed openstack/nova master: Use dict object for request_specs_dict in the _list_view https://review.opendev.org/c/openstack/nova/+/939658 | |
| 13:04:38 | sean-k-mooney | dansmith: i feel like you will care about https://etherpad.opendev.org/p/nova-2025.2-ptg#L577 | |
| 13:04:45 | sean-k-mooney | wel have an opionion | |
| 13:05:19 | sean-k-mooney | care implies a positive emotional sentement that i dont think you will express toward the idea | |
| 13:06:43 | sahid | o/ | |
| 13:07:35 | sahid | We don't have any logs in scheduler to give some information regarding the how the list is re-ordered by weighter | |
| 13:08:22 | sean-k-mooney | we do at debug | |
| 13:08:29 | sean-k-mooney | we have the weights per host | |
| 13:08:43 | sean-k-mooney | per weigher | |