| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-08 | |||
| 13:55:59 | gibi | mriedem: but I want to avoid an extra RPC bump if possible | |
| 13:57:23 | mriedem | .yeah | |
| 13:57:27 | mriedem | +W on the qos move spec | |
| 13:57:39 | gibi | mriedem: thanks | |
| 14:04:28 | alex_xu | efried: nothing more, I just want to pick a time go through the patch again | |
| 14:05:02 | efried | alex_xu: okay, thanks. Any idea who we can get to be a second +2 there? I haven't seen johnthetubaguy in a while... | |
| 14:05:42 | alex_xu | efried: good question, and we just lost ken'ichi | |
| 14:06:09 | alex_xu | efried: maybe cdent and gmann can give a review on it | |
| 14:08:24 | efried | stephenfin: +A doc restructure patch | |
| 14:09:00 | cdent | alex_xu: which? | |
| 14:09:17 | alex_xu | cdent: this one https://review.opendev.org/685181 | |
| 14:09:25 | cdent | thanks | |
| 14:09:36 | efried | cdent: you +1ed an earlier version of the test patch underneath it | |
| 14:09:53 | efried | And I was looking for your help on the fix itself before you went on vacay | |
| 14:10:00 | stephenfin | efried: Thanks (y) | |
| 14:10:22 | cdent | yeah, I've gone back to that a few different times, wandered around vaguely over the paste.ini stuff, got a bit sad about that, and then got distracted again. will gaze again now | |
| 14:10:30 | gmann | alex_xu: efried sure, I will check that | |
| 14:11:09 | efried | cdent: yeah, the paste stuff is pretty arcane, but it turns out we're handling the URL mapping bit in nova, so I was able to hack into it a bit. | |
| 14:11:24 | efried | thanks gmann | |
| 14:11:51 | efried | If this flies, there are other projects with the same problem that may need to take similar approaches. | |
| 14:11:57 | openstackgerrit | Merged openstack/nova-specs master: Support move operations with qos ports - Ussuri https://review.opendev.org/686779 | |
| 14:11:59 | efried | neutron and I think cinder | |
| 14:12:40 | cdent | during the future times of tech debt cleanup we should consider adding a wsgi entry point that doesn't use paste.ini (while keeping (perhaps deprecating) for a while the paste way) | |
| 14:12:59 | efried | cdent: I talked with lbragstad about how they cut keystone over to flask | |
| 14:13:03 | cdent | I say this as the maintainer of pastedeploy paste and pastescript: I am not actually maintaining it | |
| 14:13:08 | efried | took them a couple releases | |
| 14:13:17 | efried | but... yes. | |
| 14:14:43 | lbragstad | we piecemeal'd it because we knew we weren't going to be able to make the complete transition in a single release (but YMMV) | |
| 14:15:23 | cdent | switchign to flask would be awesome, but there are smaller steps to take if getting rid of paste.ini is the intermerdiate goal | |
| 14:15:28 | efried | dansmith: Please merge the "sev busted docs" patch at 9am your time. That's 5pm for aspiers I think, a reasonable approximation of EOD. | |
| 14:16:13 | efried | cdent: intermediate steps that would actually move the ball? Or just replace one icky thing with another icky thing? | |
| 14:16:14 | dansmith | efried: ack | |
| 14:16:21 | lbragstad | cdent yeah - i'd be curious to know how nova tries to break that work up | |
| 14:16:57 | cdent | efried: actually move the ball. all paste.ini does it assemble middleware in a configurable way. If you choose to make it not configurable, you can do the same thing in code, like what placement/deploy.py does | |
| 14:17:15 | cdent | the interior apps remain the same | |
| 14:17:26 | efried | that makes sense. Do we have any idea whether deployments count on the configurability? | |
| 14:17:34 | lbragstad | we implemented some basic plumbing that introduced flask and incrementally moved APIs over from the old paste routers to using flask | |
| 14:17:59 | efried | like, do deployments inject security sniffers or whatever? | |
| 14:18:23 | cdent | a few years ago there was some discussion of paste.ini being an official config file, but I don't know if any hard numbers were available. that's why I was suggesting we have two diferent entry points for some duration | |
| 14:18:52 | efried | oic, with a conf switch that decides which one you're using? | |
| 14:19:17 | mriedem | efried: there are deployments that inject their own middleware yes | |
| 14:19:24 | efried | cdent: community goal: "stop using paste"? | |
| 14:19:53 | efried | btw, why is it called "paste"? | |
| 14:19:56 | cdent | that will go as well as stop using wsme (which is even less maintained) | |
| 14:20:01 | mriedem | one example i know of is a middleware from inap (i think?) that translated a request to like a volume-backed flavor | |
| 14:20:11 | cdent | it's pastes things (wsgis) together | |
| 14:20:25 | cdent | but it doesn't fix my typing | |
| 14:21:21 | dansmith | I think there was also another thing called paste that did a similar thing that this was modeled after, IIRC | |
| 14:21:25 | dansmith | it's been a loong time | |
| 14:21:51 | dansmith | or maybe a "paste function" in some other middleware stack | |
| 14:22:02 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pull up compute node queries to init_host https://review.opendev.org/682680 | |
| 14:22:05 | cdent | or zope | |
| 14:22:19 | dansmith | I think removing it would be a big deal, meaning removing a big plug point for people | |
| 14:22:56 | lbragstad | +1 dansmith - iirc we asked people over and over if there was anything they *needed* paste for in their deployments | |
| 14:23:39 | dansmith | lbragstad: to be clear, I meant removing it would be a big *bad* deal for the users that make use of it | |
| 14:24:27 | efried | lbragstad: knowing zero about flask, does it also allow customization like paste does? | |
| 14:24:38 | lbragstad | no - not really (or not that i'm aware of) | |
| 14:24:59 | efried | okay, so it really would be taking a nontrivial knob away from deployers. | |
| 14:25:00 | dansmith | I don't think it does | |
| 14:25:10 | cdent | kmalloc had a plan of some kind, but I don't know if it quite happened | |
| 14:25:30 | lbragstad | cdent plan for? | |
| 14:25:35 | donnyd | well also those who may use openstack in customized ways would also be effected | |
| 14:25:51 | cdent | lbragstad: flask middleware customization by file | |
| 14:26:22 | donnyd | say I wanted to make an an integration that isn't upstream yet, paste could also be used that way | |
| 14:26:23 | cdent | I would think, that if we want to make it less configurable there would still be an option to wrap the nova wsgi application with a local custom thing: any wsgi app can be wrapped by another | |
| 14:27:16 | efried | gibi: Sorry if I'm being super dense here, but I don't see how sharing provider allocations are (even theoretically) duplicated by move_allocations | |
| 14:28:20 | gibi | efried: move_allocation takes everything that is allocated by the instance_uuid and moves it to the migration_uuid including the allocation against the sharing rp. Then when the scheduler allocates on the target host is allocates again from the sharing rp | |
| 14:28:26 | cdent | mriedem: since you seem to have a memory, does it contain a reference to recentish performance improvements when listing neutron ports? | |
| 14:29:14 | gibi | efried: if this is a migration with shared disk, it means that during the migration there will be double disk allocation but singulare disk usage | |
| 14:29:36 | efried | gibi: It looks to me as though move_allocations always deletes the allocation from the source | |
| 14:29:56 | gibi | efried: yes, then the scheduler allocates on the dest | |
| 14:30:20 | gibi | efried: when I say duble allocation I mean one piece of disk allocation by the migration_uuid and one piece of allocation by the instance_uuid | |
| 14:30:30 | efried | but I thought the migration UUID is allocating against the dest already | |
| 14:30:30 | mriedem | cdent: sorry, context? | |
| 14:31:02 | gibi | efried: migration uuid is allocates against the source, scheduler allocates against the dest for the intance_uuid as consumer | |
| 14:32:40 | cdent | mriedem: limited context, thus the bad query, trying to see if it pinged a memory for you. basically I'm looking at an internal bug where someone is complaining about 'neutron port-list' consuming a lot of time when there are many instances, and that pinged a memory in me of reading either a bug fix or patch on the nova side related to listing ports being slow | |
| 14:32:55 | efried | And you're saying that during (2) we would have duplicate sharing provider allocations in theory. | |
| 14:32:55 | efried | gibi: ohhh, so when we're doing a migration, we (1) move_allocations(source_inst/source_host => mig_inst/source_host), and then (2) claim(dest_inst/dest_host), and then (3) delete(mig_inst/source_host) | |
| 14:33:10 | openstackgerrit | Merged openstack/nova-specs master: tox -e fast-specs https://review.opendev.org/673356 | |
| 14:33:17 | mriedem | cdent: purely a neutron perf bug? | |
| 14:33:27 | mriedem | or also listing servers being slow b/c of listing ports (security groups)? | |
| 14:33:36 | mriedem | i know of a thing, sec | |
| 14:33:37 | gibi | efried: yes. and for sharing providers this double up is not needed | |
| 14:33:49 | efried | gibi: got it, thank you. | |
| 14:34:08 | mriedem | cdent: is more on the nova server listing side but it's a perf thing https://bugs.launchpad.net/nova/+bug/1567655 | |
| 14:34:09 | openstack | Launchpad bug 1567655 in OpenStack Compute (nova) "500 error when trying to list instances and neutron-server is down" [Medium,Confirmed] | |
| 14:34:11 | cdent | mriedem: I think the thing I'm remember on the nova side was listing servers being slow because neutron being slow and I wanted to reference that thing (for more context) if I could find it, but I couldnt but though tyou might remember | |
| 14:34:15 | gibi | efried: if you can suggest a better wording for the code comment in my patch please do so | |
| 14:34:29 | cdent | see, you did. thank you. | |
| 14:34:35 | cdent | let's see if that's of any use | |
| 14:34:45 | mriedem | cdent: comment 16 is the summary on the perf part of it | |
| 14:34:58 | efried | gibi: It probably makes sense in context, for someone who has that flow (1)(2)(3) in their head. I'll just leave a note on the review for future me and +2. Thanks for bearing with me. | |
| 14:35:14 | gibi | efried: thanks | |
| 14:35:32 | cdent | mriedem: yeah, thanks that helps | |
| 14:36:14 | mriedem | unrelated to listing servers, there are known inefficiencies down in the nova/network/neutronv2/api.py code where nova makes a bunch of small targeted requests to neutron for resources rather than bigger batch requests | |
| 14:37:25 | mriedem | stuff like https://review.opendev.org/#/c/465792/ | |
| 14:38:30 | cdent | excellent, thanks | |
| 14:38:53 | mriedem | there is an abandoned wip patch related to that from someone at huawei that did a much more extensive refactor to batch up the queries | |
| 14:39:16 | cdent | shame about that | |
| 14:39:22 | mriedem | well, it was pretty crazy | |