| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-08 | |||
| 14:16:14 | dansmith | efried: ack | |
| 14:16:21 | lbragstad | cdent yeah - i'd be curious to know how nova tries to break that work up | |
| 14:16:57 | cdent | efried: actually move the ball. all paste.ini does it assemble middleware in a configurable way. If you choose to make it not configurable, you can do the same thing in code, like what placement/deploy.py does | |
| 14:17:15 | cdent | the interior apps remain the same | |
| 14:17:26 | efried | that makes sense. Do we have any idea whether deployments count on the configurability? | |
| 14:17:34 | lbragstad | we implemented some basic plumbing that introduced flask and incrementally moved APIs over from the old paste routers to using flask | |
| 14:17:59 | efried | like, do deployments inject security sniffers or whatever? | |
| 14:18:23 | cdent | a few years ago there was some discussion of paste.ini being an official config file, but I don't know if any hard numbers were available. that's why I was suggesting we have two diferent entry points for some duration | |
| 14:18:52 | efried | oic, with a conf switch that decides which one you're using? | |
| 14:19:17 | mriedem | efried: there are deployments that inject their own middleware yes | |
| 14:19:24 | efried | cdent: community goal: "stop using paste"? | |
| 14:19:53 | efried | btw, why is it called "paste"? | |
| 14:19:56 | cdent | that will go as well as stop using wsme (which is even less maintained) | |
| 14:20:01 | mriedem | one example i know of is a middleware from inap (i think?) that translated a request to like a volume-backed flavor | |
| 14:20:11 | cdent | it's pastes things (wsgis) together | |
| 14:20:25 | cdent | but it doesn't fix my typing | |
| 14:21:21 | dansmith | I think there was also another thing called paste that did a similar thing that this was modeled after, IIRC | |
| 14:21:25 | dansmith | it's been a loong time | |
| 14:21:51 | dansmith | or maybe a "paste function" in some other middleware stack | |
| 14:22:02 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pull up compute node queries to init_host https://review.opendev.org/682680 | |
| 14:22:05 | cdent | or zope | |
| 14:22:19 | dansmith | I think removing it would be a big deal, meaning removing a big plug point for people | |
| 14:22:56 | lbragstad | +1 dansmith - iirc we asked people over and over if there was anything they *needed* paste for in their deployments | |
| 14:23:39 | dansmith | lbragstad: to be clear, I meant removing it would be a big *bad* deal for the users that make use of it | |
| 14:24:27 | efried | lbragstad: knowing zero about flask, does it also allow customization like paste does? | |
| 14:24:38 | lbragstad | no - not really (or not that i'm aware of) | |
| 14:24:59 | efried | okay, so it really would be taking a nontrivial knob away from deployers. | |
| 14:25:00 | dansmith | I don't think it does | |
| 14:25:10 | cdent | kmalloc had a plan of some kind, but I don't know if it quite happened | |
| 14:25:30 | lbragstad | cdent plan for? | |
| 14:25:35 | donnyd | well also those who may use openstack in customized ways would also be effected | |
| 14:25:51 | cdent | lbragstad: flask middleware customization by file | |
| 14:26:22 | donnyd | say I wanted to make an an integration that isn't upstream yet, paste could also be used that way | |
| 14:26:23 | cdent | I would think, that if we want to make it less configurable there would still be an option to wrap the nova wsgi application with a local custom thing: any wsgi app can be wrapped by another | |
| 14:27:16 | efried | gibi: Sorry if I'm being super dense here, but I don't see how sharing provider allocations are (even theoretically) duplicated by move_allocations | |
| 14:28:20 | gibi | efried: move_allocation takes everything that is allocated by the instance_uuid and moves it to the migration_uuid including the allocation against the sharing rp. Then when the scheduler allocates on the target host is allocates again from the sharing rp | |
| 14:28:26 | cdent | mriedem: since you seem to have a memory, does it contain a reference to recentish performance improvements when listing neutron ports? | |
| 14:29:14 | gibi | efried: if this is a migration with shared disk, it means that during the migration there will be double disk allocation but singulare disk usage | |
| 14:29:36 | efried | gibi: It looks to me as though move_allocations always deletes the allocation from the source | |
| 14:29:56 | gibi | efried: yes, then the scheduler allocates on the dest | |
| 14:30:20 | gibi | efried: when I say duble allocation I mean one piece of disk allocation by the migration_uuid and one piece of allocation by the instance_uuid | |
| 14:30:30 | mriedem | cdent: sorry, context? | |
| 14:30:30 | efried | but I thought the migration UUID is allocating against the dest already | |
| 14:31:02 | gibi | efried: migration uuid is allocates against the source, scheduler allocates against the dest for the intance_uuid as consumer | |
| 14:32:40 | cdent | mriedem: limited context, thus the bad query, trying to see if it pinged a memory for you. basically I'm looking at an internal bug where someone is complaining about 'neutron port-list' consuming a lot of time when there are many instances, and that pinged a memory in me of reading either a bug fix or patch on the nova side related to listing ports being slow | |
| 14:32:55 | efried | gibi: ohhh, so when we're doing a migration, we (1) move_allocations(source_inst/source_host => mig_inst/source_host), and then (2) claim(dest_inst/dest_host), and then (3) delete(mig_inst/source_host) | |
| 14:32:55 | efried | And you're saying that during (2) we would have duplicate sharing provider allocations in theory. | |
| 14:33:10 | openstackgerrit | Merged openstack/nova-specs master: tox -e fast-specs https://review.opendev.org/673356 | |
| 14:33:17 | mriedem | cdent: purely a neutron perf bug? | |
| 14:33:27 | mriedem | or also listing servers being slow b/c of listing ports (security groups)? | |
| 14:33:36 | mriedem | i know of a thing, sec | |
| 14:33:37 | gibi | efried: yes. and for sharing providers this double up is not needed | |
| 14:33:49 | efried | gibi: got it, thank you. | |
| 14:34:08 | mriedem | cdent: is more on the nova server listing side but it's a perf thing https://bugs.launchpad.net/nova/+bug/1567655 | |
| 14:34:09 | openstack | Launchpad bug 1567655 in OpenStack Compute (nova) "500 error when trying to list instances and neutron-server is down" [Medium,Confirmed] | |
| 14:34:11 | cdent | mriedem: I think the thing I'm remember on the nova side was listing servers being slow because neutron being slow and I wanted to reference that thing (for more context) if I could find it, but I couldnt but though tyou might remember | |
| 14:34:15 | gibi | efried: if you can suggest a better wording for the code comment in my patch please do so | |
| 14:34:29 | cdent | see, you did. thank you. | |
| 14:34:35 | cdent | let's see if that's of any use | |
| 14:34:45 | mriedem | cdent: comment 16 is the summary on the perf part of it | |
| 14:34:58 | efried | gibi: It probably makes sense in context, for someone who has that flow (1)(2)(3) in their head. I'll just leave a note on the review for future me and +2. Thanks for bearing with me. | |
| 14:35:14 | gibi | efried: thanks | |
| 14:35:32 | cdent | mriedem: yeah, thanks that helps | |
| 14:36:14 | mriedem | unrelated to listing servers, there are known inefficiencies down in the nova/network/neutronv2/api.py code where nova makes a bunch of small targeted requests to neutron for resources rather than bigger batch requests | |
| 14:37:25 | mriedem | stuff like https://review.opendev.org/#/c/465792/ | |
| 14:38:30 | cdent | excellent, thanks | |
| 14:38:53 | mriedem | there is an abandoned wip patch related to that from someone at huawei that did a much more extensive refactor to batch up the queries | |
| 14:39:16 | cdent | shame about that | |
| 14:39:22 | mriedem | well, it was pretty crazy | |
| 14:39:34 | mriedem | they are probably running it in their public cloud though... | |
| 14:39:50 | mriedem | this: https://review.opendev.org/#/c/457845/ | |
| 14:39:54 | cdent | I bet, but if it could someday see the light of day (in a better form?) that would be grand | |
| 14:40:59 | mriedem | if someone were serious about that effort, i think we'd want some before and after profiling numbers like what you did in placement to justify the effort | |
| 14:41:31 | mriedem | i know windriver cared about that stuff because they have users with vms that have like 16 ports attached | |
| 14:41:44 | mriedem | *nfv users | |
| 14:42:36 | cdent | I suspect that it would be of benefit to vmware with their n-thousand instances in one nova-compute mess, but...human resources... | |
| 14:45:22 | gibi | stephenfin: quick question in https://review.opendev.org/#/c/686801/3//COMMIT_MSG@9 | |
| 14:45:25 | mriedem | alternatively neutron grows an api that is essentially a view | |
| 14:45:42 | mriedem | i think ^ was bantered about since icehouse days | |
| 14:45:51 | stephenfin | gibi: Yup, that's what I meant. My bad | |
| 14:46:17 | gibi | stephenfin: no worries, I just wanted to 100% sure you did what you wanted to do | |
| 14:46:24 | stephenfin | all good :) | |
| 14:46:39 | gibi | :) | |
| 14:56:41 | mriedem | cdent: i've restored https://review.opendev.org/#/c/465792/ to see if i can rebase that since apparently i got it working with kevinbenton's help at one point, and then see if i can tease out some before/after times in a local devstack | |
| 14:56:57 | mriedem | my guess is the server would need to have a few ports attached to notice any different | |
| 14:56:59 | mriedem | *difference | |
| 15:01:49 | dansmith | mriedem: I was under the impression from your doc that "tox -e api_samples" would create samples I used in the new test I added | |
| 15:02:39 | mriedem | s/your/the/ | |
| 15:02:55 | mriedem | it should generate samples under tox/docs/api_samples yeah | |
| 15:03:00 | mriedem | for the functional api samples tests you run | |
| 15:03:11 | dansmith | do I need to create the microversion dir or something? | |
| 15:03:17 | mriedem | yeah i think so | |
| 15:03:21 | dansmith | tox -eapi_samples didn't seem to actually run anything though, actually | |
| 15:03:35 | mriedem | b/c it's tox -e api-samples | |
| 15:03:36 | mriedem | not api_samples | |
| 15:03:40 | mriedem | is that a typo in the doc? | |
| 15:03:45 | dansmith | gah | |
| 15:03:51 | dansmith | no | |
| 15:04:12 | dansmith | typo on my end, coupled with I guess I still have that old tox that doesn't complain | |
| 15:04:13 | mriedem | you must have a busted version of tox because that would normally fail if you specify an env that doesn't exist | |