Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-12-05
15:12:08 mriedem blindly retrying on *any* exception when doing something like deleting an attachment makes me nervous because if cinder returns a 404 because the original request completed and then we keep retrying, we're just going to keep retrying and getting 404s until we exhaust our retries
15:12:21 stephenfin good point
15:12:25 lyarwood stephenfin: no specific reason other than a personal dislike of hardcoded retry values tbh
15:12:27 mriedem iow, can we scope those retries to just 500 errors?
15:12:58 stephenfin and I have a personal dislike of knobs that no one ever touches
15:13:05 stephenfin Unstoppable force, meet immovable object :)
15:13:31 lyarwood Meets the guy with +2 over the guy with +1 so you win
15:13:53 lyarwood mriedem: yeah that's fair, I think a previous PS did previously and I've likely missed that cinder_exception.ClientException covers both?
15:14:48 lyarwood yeah it's the base, gah my bad.
15:15:26 mriedem right so https://github.com/openstack/python-cinderclient/blob/5.0.0/cinderclient/exceptions.py#L242 for 404 will create a NotFound(ClientException) and we'd retry on that
15:15:49 mriedem so we need to scope the retries to ClientExceptions with code==500 only imo
15:16:36 mriedem would also be nice to eventually add long_rpc_timeout to cinder for apis like this that have been known to take awhile
15:16:43 mriedem i know we've seen attachment delete timeout in the gate
15:16:57 mriedem surely red hat has some people working on cinder yeah?
15:17:37 lyarwood yup we do, I can follow up with them about that
15:17:46 mriedem https://bugs.launchpad.net/cinder/+bug/1763712 is where i suggested using long_rpc_timeout in cinder
15:17:47 openstack Launchpad bug 1763712 in Cinder "Unable to update the attachment.: MessagingTimeout" [Medium,Confirmed]
15:17:59 mriedem it shouldn't be that hard to add since the cinder rpc stuff is likely very similar to nova's
15:32:22 mriedem gibi: on https://review.opendev.org/#/c/693248/ can't we just hard-code the server created to use the net0 network - that's the one that has the port with the qos allocations
15:34:48 mriedem oh well i guess shared != external
15:35:15 mriedem so we got the shared public external network and nova puked because by default non-admins can't create servers on external networks
15:39:38 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Cache security group driver https://review.opendev.org/697505
15:44:16 openstack bug 1849657 in OpenStack Compute (nova) train " allocation key is missing from the binding:profile of the neutron qos port when the server is created by a non-admin user" [Medium,Fix committed] https://launchpad.net/bugs/1849657 - Assigned to Balazs Gibizer (balazs-gibizer)
15:44:16 openstackgerrit Matt Riedemann proposed openstack/nova master: Regression test for bug 1849657 https://review.opendev.org/693248
15:49:01 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: log when loading security group driver https://review.opendev.org/652783
15:53:54 efried mriedem: ack, thanks for that.
15:54:02 efried my cold has decided to kill me, so I may be sparse today.
15:54:46 mriedem RIP
15:55:37 mriedem efried: fyi i've got a thing in the nova meeting agenda for today but likely not going to be able to attend, it's just a question about if a spec is needed for that filter i'm working on
15:56:05 efried ack
15:56:17 mriedem also, the price is right is on in 4 minutes
15:56:31 mriedem good time to get on the couch with a wubby
16:00:45 efried stephenfin: https://review.opendev.org/#/c/696507/ if you want to update that commit message quick I'll fast approve, no need to rebase anything.
16:01:32 efried do we still say things like $show is on at $time? Who watches anything on a broadcast schedule?
16:02:17 stephenfin okidok
16:03:36 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove SG tests that don't apply to neutron https://review.opendev.org/696507
16:03:44 stephenfin efried: ^
16:04:18 efried stephenfin: +A, thanks for accommodating my pickiness.
16:04:29 stephenfin thanks for the review (y)
16:04:50 mordred efried: I think we maybe contracted the same plague
16:05:09 mordred mriedem: do we need an sdk release for the ironic patch we landed for you?
16:05:21 efried I suspect I actually got two things back to back. Which isn't supposed to happen.
16:07:11 mriedem mordred: at some point, but the nova blueprint that needs it isn't approved yet so it's not a rush
16:07:39 mriedem efried: i wouldn't dvr the price is right because i only watch it like twice a year
16:08:29 mriedem about the same amount of times i buy bacon - usually in the same week
16:08:49 mriedem ah the holidays, a time for bacon and day time tv game shows
16:10:59 efried wow, I buy bacon weekly, in multiple pounds.
16:12:31 mordred my biggest issue with bacon is that it's never possible to buy enough
16:13:19 mriedem no one else in my house will eat it (now that mortimer the dog is dead anyway) so i'm not making that smelly mess for myself every day
16:14:03 efried I make the whole mess at once and bag it. Bacon is a food that loses almost nothing reheated.
16:14:23 mriedem reheated in the microwave?
16:14:27 mriedem blasphemy
16:31:02 dansmith not blasphemy, plus it means bacon more often
16:31:06 dansmith BACON AT EVERY MEAL (tm)
16:32:31 efried Okay, I updated the meeting agenda. I'm going to go be miserable somewhere for a while.
16:34:59 openstackgerrit Matt Riedemann proposed openstack/nova stable/stein: Improve metadata server performance with large security groups https://review.opendev.org/695925
16:40:30 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Improve metadata server performance with large security groups https://review.opendev.org/697517
16:41:31 melwitt mriedem: I went out on a limb and proposed a backport to queens that landed in rocky, for cleaning up placement consumer records that no longer have allocations https://review.opendev.org/697398 I had to modify it a lot because the code was in different files in queens
16:42:46 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Improve metadata server performance with large security groups https://review.opendev.org/697518
16:43:04 melwitt not sure whether that's going to fly but I figured I'd try because it's a pretty big unbounded table grow that otherwise has to be cleaned up manually
16:45:48 mriedem queens is EM so that's pretty low priority to address but fine to post if you're doing it for downstream anyway and if others want to use it
16:46:37 melwitt ah, I forgot it's EM. ok
16:46:49 melwitt understood
16:52:13 mriedem dansmith: how do you feel about backporting this? https://review.opendev.org/#/c/696953/
16:52:28 mriedem seems ok since it's tied to a long-standing bug but wanted a 2nd opinion
16:53:04 dansmith mriedem: yeah, I mean... as long as the old version has a sufficient oslo.messaging and the config knob, I can't really think of any reason not to do tose
16:53:05 dansmith *those
16:53:47 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Improve metadata server performance with large security groups https://review.opendev.org/697523
16:56:25 mriedem lyarwood: i'd be +2 on this https://review.opendev.org/#/c/696953/ but i'm not sure how pedantic you want to be about using the correct source commit hash
16:57:21 mriedem given train and stein had conflicts which you've already resolved i'm assuming you don't want to redo all of that
16:59:35 mriedem nvm i just +2ed
17:01:05 mriedem dansmith: you want to get this https://review.opendev.org/#/c/696339/ and this then https://review.opendev.org/696953
17:03:22 dansmith yar
17:06:31 lyarwood mriedem: sorry was on a call, I really don't care about the source commit hash with backports given the Change IDs tbh. That said I have no idea how I ended up using the wrong one for this.
17:06:34 lyarwood mriedem: and thanks
17:33:14 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: rt: only map compute node if we created it https://review.opendev.org/676463
17:54:48 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Fix listing deleted servers with a marker https://review.opendev.org/690747
17:54:48 openstack bug 1849409 in OpenStack Compute (nova) queens "openstack server list --deleted --limit -1 hangs" [High,In progress] https://launchpad.net/bugs/1849409 - Assigned to Matt Riedemann (mriedem)
17:54:48 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Add functional regression test for bug 1849409 https://review.opendev.org/690746
17:57:21 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Remove 'get_invalid_image' https://review.opendev.org/697454
17:57:21 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182
17:57:22 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add unified '_(build|create)_flavor' helper functions https://review.opendev.org/697538
17:57:22 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add unified '_build_server' helper function https://review.opendev.org/697537
17:57:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add '_create_server' helper https://review.opendev.org/697540
17:57:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add '_delete_server' to 'InstanceHelperMixin' https://review.opendev.org/697539
18:05:58 stephenfin sean-k-mooney: One comment on https://review.opendev.org/#/c/674072/
18:16:35 stephenfin sean-k-mooney: also on https://review.opendev.org/#/c/687957/
18:16:59 stephenfin and with that...
18:35:49 sean-k-mooney stephenfin: yep ill take a look and adress them today. just back form dinner so ill work on that shortly
18:50:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Add resource provider allocation unset example to troubleshooting doc https://review.opendev.org/696582
19:17:00 efried mriedem, smcginnis, jungleboyj: Any ideas on how to make sure we've tested https://review.opendev.org/697134 through os-brick?
19:19:39 mriedem efried: i commented but tl;dr i can't think of anything
19:20:01 mriedem this is assuming that all of os-brick is using privsep and not any rootwrap helpers for some of the backends in os-brick somewhere
19:20:13 mriedem we can see that it's fine for lvm and rbd though
19:20:38 mriedem the libvirt driver uses os-brick, the ceph job uses the rbd volume type and backend, and all other jobs are using lvm
19:21:09 mriedem if there is something else in brick not using privsep then we likely won't see it here, but once we land this i'd think 3rd party CIs in cinder would be failing and alerting to cinder team
19:21:34 mriedem though if cinder's 3rd party ci's are anything like ours, there are probably many that fail often so people don't pay attention
19:22:43 mriedem the quobyte ci might be a decent indication and it failed but it also didn't report anything
19:22:50 mriedem kaisers: ^
19:24:12 efried thanks for the look mriedem

Earlier   Later