Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-10-08
14:12:59 efried cdent: I talked with lbragstad about how they cut keystone over to flask
14:13:03 cdent I say this as the maintainer of pastedeploy paste and pastescript: I am not actually maintaining it
14:13:08 efried took them a couple releases
14:13:17 efried but... yes.
14:14:43 lbragstad we piecemeal'd it because we knew we weren't going to be able to make the complete transition in a single release (but YMMV)
14:15:23 cdent switchign to flask would be awesome, but there are smaller steps to take if getting rid of paste.ini is the intermerdiate goal
14:15:28 efried dansmith: Please merge the "sev busted docs" patch at 9am your time. That's 5pm for aspiers I think, a reasonable approximation of EOD.
14:16:13 efried cdent: intermediate steps that would actually move the ball? Or just replace one icky thing with another icky thing?
14:16:14 dansmith efried: ack
14:16:21 lbragstad cdent yeah - i'd be curious to know how nova tries to break that work up
14:16:57 cdent efried: actually move the ball. all paste.ini does it assemble middleware in a configurable way. If you choose to make it not configurable, you can do the same thing in code, like what placement/deploy.py does
14:17:15 cdent the interior apps remain the same
14:17:26 efried that makes sense. Do we have any idea whether deployments count on the configurability?
14:17:34 lbragstad we implemented some basic plumbing that introduced flask and incrementally moved APIs over from the old paste routers to using flask
14:17:59 efried like, do deployments inject security sniffers or whatever?
14:18:23 cdent a few years ago there was some discussion of paste.ini being an official config file, but I don't know if any hard numbers were available. that's why I was suggesting we have two diferent entry points for some duration
14:18:52 efried oic, with a conf switch that decides which one you're using?
14:19:17 mriedem efried: there are deployments that inject their own middleware yes
14:19:24 efried cdent: community goal: "stop using paste"?
14:19:53 efried btw, why is it called "paste"?
14:19:56 cdent that will go as well as stop using wsme (which is even less maintained)
14:20:01 mriedem one example i know of is a middleware from inap (i think?) that translated a request to like a volume-backed flavor
14:20:11 cdent it's pastes things (wsgis) together
14:20:25 cdent but it doesn't fix my typing
14:21:21 dansmith I think there was also another thing called paste that did a similar thing that this was modeled after, IIRC
14:21:25 dansmith it's been a loong time
14:21:51 dansmith or maybe a "paste function" in some other middleware stack
14:22:02 openstackgerrit Balazs Gibizer proposed openstack/nova master: Pull up compute node queries to init_host https://review.opendev.org/682680
14:22:05 cdent or zope
14:22:19 dansmith I think removing it would be a big deal, meaning removing a big plug point for people
14:22:56 lbragstad +1 dansmith - iirc we asked people over and over if there was anything they *needed* paste for in their deployments
14:23:39 dansmith lbragstad: to be clear, I meant removing it would be a big *bad* deal for the users that make use of it
14:24:27 efried lbragstad: knowing zero about flask, does it also allow customization like paste does?
14:24:38 lbragstad no - not really (or not that i'm aware of)
14:24:59 efried okay, so it really would be taking a nontrivial knob away from deployers.
14:25:00 dansmith I don't think it does
14:25:10 cdent kmalloc had a plan of some kind, but I don't know if it quite happened
14:25:30 lbragstad cdent plan for?
14:25:35 donnyd well also those who may use openstack in customized ways would also be effected
14:25:51 cdent lbragstad: flask middleware customization by file
14:26:22 donnyd say I wanted to make an an integration that isn't upstream yet, paste could also be used that way
14:26:23 cdent I would think, that if we want to make it less configurable there would still be an option to wrap the nova wsgi application with a local custom thing: any wsgi app can be wrapped by another
14:27:16 efried gibi: Sorry if I'm being super dense here, but I don't see how sharing provider allocations are (even theoretically) duplicated by move_allocations
14:28:20 gibi efried: move_allocation takes everything that is allocated by the instance_uuid and moves it to the migration_uuid including the allocation against the sharing rp. Then when the scheduler allocates on the target host is allocates again from the sharing rp
14:28:26 cdent mriedem: since you seem to have a memory, does it contain a reference to recentish performance improvements when listing neutron ports?
14:29:14 gibi efried: if this is a migration with shared disk, it means that during the migration there will be double disk allocation but singulare disk usage
14:29:36 efried gibi: It looks to me as though move_allocations always deletes the allocation from the source
14:29:56 gibi efried: yes, then the scheduler allocates on the dest
14:30:20 gibi efried: when I say duble allocation I mean one piece of disk allocation by the migration_uuid and one piece of allocation by the instance_uuid
14:30:30 efried but I thought the migration UUID is allocating against the dest already
14:30:30 mriedem cdent: sorry, context?
14:31:02 gibi efried: migration uuid is allocates against the source, scheduler allocates against the dest for the intance_uuid as consumer
14:32:40 cdent mriedem: limited context, thus the bad query, trying to see if it pinged a memory for you. basically I'm looking at an internal bug where someone is complaining about 'neutron port-list' consuming a lot of time when there are many instances, and that pinged a memory in me of reading either a bug fix or patch on the nova side related to listing ports being slow
14:32:55 efried And you're saying that during (2) we would have duplicate sharing provider allocations in theory.
14:32:55 efried gibi: ohhh, so when we're doing a migration, we (1) move_allocations(source_inst/source_host => mig_inst/source_host), and then (2) claim(dest_inst/dest_host), and then (3) delete(mig_inst/source_host)
14:33:10 openstackgerrit Merged openstack/nova-specs master: tox -e fast-specs https://review.opendev.org/673356
14:33:17 mriedem cdent: purely a neutron perf bug?
14:33:27 mriedem or also listing servers being slow b/c of listing ports (security groups)?
14:33:36 mriedem i know of a thing, sec
14:33:37 gibi efried: yes. and for sharing providers this double up is not needed
14:33:49 efried gibi: got it, thank you.
14:34:08 mriedem cdent: is more on the nova server listing side but it's a perf thing https://bugs.launchpad.net/nova/+bug/1567655
14:34:09 openstack Launchpad bug 1567655 in OpenStack Compute (nova) "500 error when trying to list instances and neutron-server is down" [Medium,Confirmed]
14:34:11 cdent mriedem: I think the thing I'm remember on the nova side was listing servers being slow because neutron being slow and I wanted to reference that thing (for more context) if I could find it, but I couldnt but though tyou might remember
14:34:15 gibi efried: if you can suggest a better wording for the code comment in my patch please do so
14:34:29 cdent see, you did. thank you.
14:34:35 cdent let's see if that's of any use
14:34:45 mriedem cdent: comment 16 is the summary on the perf part of it
14:34:58 efried gibi: It probably makes sense in context, for someone who has that flow (1)(2)(3) in their head. I'll just leave a note on the review for future me and +2. Thanks for bearing with me.
14:35:14 gibi efried: thanks
14:35:32 cdent mriedem: yeah, thanks that helps
14:36:14 mriedem unrelated to listing servers, there are known inefficiencies down in the nova/network/neutronv2/api.py code where nova makes a bunch of small targeted requests to neutron for resources rather than bigger batch requests
14:37:25 mriedem stuff like https://review.opendev.org/#/c/465792/
14:38:30 cdent excellent, thanks
14:38:53 mriedem there is an abandoned wip patch related to that from someone at huawei that did a much more extensive refactor to batch up the queries
14:39:16 cdent shame about that
14:39:22 mriedem well, it was pretty crazy
14:39:34 mriedem they are probably running it in their public cloud though...
14:39:50 mriedem this: https://review.opendev.org/#/c/457845/
14:39:54 cdent I bet, but if it could someday see the light of day (in a better form?) that would be grand
14:40:59 mriedem if someone were serious about that effort, i think we'd want some before and after profiling numbers like what you did in placement to justify the effort
14:41:31 mriedem i know windriver cared about that stuff because they have users with vms that have like 16 ports attached
14:41:44 mriedem *nfv users
14:42:36 cdent I suspect that it would be of benefit to vmware with their n-thousand instances in one nova-compute mess, but...human resources...
14:45:22 gibi stephenfin: quick question in https://review.opendev.org/#/c/686801/3//COMMIT_MSG@9
14:45:25 mriedem alternatively neutron grows an api that is essentially a view
14:45:42 mriedem i think ^ was bantered about since icehouse days
14:45:51 stephenfin gibi: Yup, that's what I meant. My bad
14:46:17 gibi stephenfin: no worries, I just wanted to 100% sure you did what you wanted to do
14:46:24 stephenfin all good :)
14:46:39 gibi :)
14:56:41 mriedem cdent: i've restored https://review.opendev.org/#/c/465792/ to see if i can rebase that since apparently i got it working with kevinbenton's help at one point, and then see if i can tease out some before/after times in a local devstack
14:56:57 mriedem my guess is the server would need to have a few ports attached to notice any different
14:56:59 mriedem *difference
15:01:49 dansmith mriedem: I was under the impression from your doc that "tox -e api_samples" would create samples I used in the new test I added
15:02:39 mriedem s/your/the/
15:02:55 mriedem it should generate samples under tox/docs/api_samples yeah
15:03:00 mriedem for the functional api samples tests you run
15:03:11 dansmith do I need to create the microversion dir or something?
15:03:17 mriedem yeah i think so

Earlier   Later