Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-04-25
15:16:04 mriedem but caching scheduler doesn't use placement
15:16:41 openstackgerrit Daniel Pawlik proposed openstack/nova master: Added kvm_hidden feature https://review.openstack.org/459753
15:17:50 bauzas mriedem: okay, because we don't fully fetch the large list of hosts, got it
15:18:09 bauzas (in the filter scheduler)
15:18:24 sean-k-mooney the caching scheduler solve a part of scheduler performance but with the placement api and schduler claim you would nolonger have race condtions in the scheduler and could scaler it liniearly horizontally by adding more schdedulers
15:18:26 mriedem well and filters in db
15:18:41 mriedem bauzas: johnthetubaguy + osic said filter scheduler + placement was still slower than caching scheduler because of the retries
15:18:46 mriedem due to not claiming in the scheduler
15:19:16 mriedem so if we pick the correct host the first time in the scheduler by doing the correct claim, we should mitigate the need for retries
15:19:27 mriedem which is also why i nag about not relying on retries in the claims in scheduler spec
15:19:28 bauzas okay
15:19:31 mriedem or assuming it will work, or be there
15:19:34 mriedem it == retries
15:20:05 openstackgerrit Sylvain Bauza proposed openstack/nova-specs master: Claims in the scheduler https://review.openstack.org/437424
15:20:28 bauzas mriedem: dansmith: updated the spec before I forgot to remove an unnecessary work item ^
15:20:32 bauzas jaypipes: too ^
15:21:33 sean-k-mooney mriedem: +1 the claims process needs to be independent of (not depend on ) retries.
15:21:53 openstackgerrit Gábor Antal proposed openstack/nova master: Transform instance.volume_detach notification https://review.openstack.org/408676
15:22:26 mriedem sean-k-mooney: until we pick the wrong host b/c we didn't know about all of your crazy extra specs and aggregate requirements for NFV :)
15:23:19 openstackgerrit sahid proposed openstack/nova master: numa: fix an uninitialized local variable https://review.openstack.org/458848
15:23:37 sean-k-mooney mriedem: ofcourse life would be so much less interesting without the nfv curve ball but nested resource provides and traits should fix that
15:24:44 bauzas sean-k-mooney: mriedem: anyway, given we only check mem/cpu/disk and not other resource classes in the placement engine, we still need to accept reschedules...
15:24:56 bauzas until we at least have nested RPs
15:25:00 bauzas for NUMA resources
15:25:49 sean-k-mooney bauzas: im reading the spec now but at what point is the claim made? before or after the filters run?
15:25:59 mriedem sean-k-mooney: after
15:26:04 dansmith sean-k-mooney: has to be after, else we don't know what to claim :)
15:26:27 sean-k-mooney ya so it should be after in the weigher stage right
15:26:31 dansmith mriedem: are we holding off on approving this until after forum, or should we just go ahead and merge and handle any feedback as an amendment?
15:26:45 dansmith sean-k-mooney: it's after weighers too
15:27:12 bauzas sean-k-mooney: tbc, it will probably be in the consume_from_request() method or somewhere close to it
15:27:14 dansmith sean-k-mooney: it's at the point at which we pick a host, but before we ask that host to build (whereas currently it's after the host has started to build)
15:27:50 bauzas because we already somehow "claim" the resource in the in-memory HostState scheduler manager
15:27:52 sean-k-mooney dansmith: bauzas that makes sense.
15:28:24 sean-k-mooney you should not need to rescudle for nfv requirements then as you have already validated that you can provided them
15:29:05 mriedem dansmith: i'm not intentionally holding it up until after the forum. i wanted to get agreement on the plan prior to the forum, start POC code, and present the plan at the forum to get feedback from ops.
15:29:05 dansmith sean-k-mooney: well, since we're not claiming the numa type resources in that claim, we'll still be able to race with filters choosing the same thing for multiple concurrent builds
15:29:12 mriedem dansmith: i just haven't read the latest revision yet
15:29:17 dansmith mriedem: yeah, I'd really like to see code before forum
15:29:38 sean-k-mooney dansmith: yes that is true
15:29:39 dansmith mriedem: okay I should take a pass through it.. maybe we can merge the spec this week and get some code going before forum if you're cool with it
15:30:00 mriedem i am cool with progress yes
15:30:11 mriedem remember, i like rush
15:30:38 sean-k-mooney dansmith: you should be able to vailidate that before calling to the host though. ie. if you can claim the numa page allocation abourt and retry
15:31:10 sean-k-mooney mriedem: is that late abort and retry the issue you were commenting on?
15:31:39 mriedem sean-k-mooney: we don't have allocations for numa yet
15:31:41 mriedem in placement
15:31:58 mriedem i believe that's nested resource providers territory
15:32:13 sean-k-mooney oh ok i understand the gap now. ya that was covered by nested providers
15:32:43 mriedem so if you're an NFV shop and we don't have nested resource providers yet, then you'll need to enable retries yes
15:32:56 mriedem which means you're not using the whole cells v2 superconductor deployment model
15:33:04 bauzas mriedem: dansmith: I can start pushing code, fersür
15:33:48 bauzas the spec, IHMO, is in a good shape with good work items, so that's a cool start
15:34:08 bauzas now that said, I need to disappear for now
15:34:18 bauzas ttyl folks
15:39:08 openstackgerrit Daniel Pawlik proposed openstack/nova master: Added kvm_hidden feature https://review.openstack.org/459753
15:40:15 dansmith sean-k-mooney: if we ran filters twice or something maybe
15:41:01 openstackgerrit Gábor Antal proposed openstack/nova master: Transform instance.volume_detach notification https://review.openstack.org/408676
15:42:19 openstackgerrit Chris Dent proposed openstack/nova master: Register osapi_compute when nova-api is wsgi https://review.openstack.org/457283
15:42:20 openstackgerrit Chris Dent proposed openstack/nova master: Add pbr-installed wsgi application for metadata api https://review.openstack.org/459413
15:46:22 openstackgerrit Gábor Antal proposed openstack/nova master: Transform instance.volume_attach.error notification https://review.openstack.org/455801
15:52:24 sean-k-mooney dansmith: perhaps though honestly if the only main gap is the nfv constartisn of numa,cpu pinning,hugepages and pci devices it sounds like that will be addressed in time by nested resouce providers.
15:52:41 dansmith sean-k-mooney: indeed, that's the plan
15:52:58 sean-k-mooney dansmith: a validate call to the filters that accpeted a claim would also work but its a lot of work that may not be worth it
15:53:17 dansmith sean-k-mooney: yeah, I'd rather focus on the right fix
15:53:35 mriedem i haven't thought yet about how we replace the affinity filters with placement
15:54:03 mriedem i guess sum the reqs for all members of an affinity group and find a provider for them
15:54:23 sean-k-mooney the affinity filters in general or the server group affinity filters
15:54:27 dansmith mriedem: needs more placement support I think.. jaypipes has waxed poetic about it in the past
15:54:29 mriedem server group
15:54:44 sean-k-mooney mriedem: traits maybe?
15:55:00 mriedem anti-affinity, you could get RPs where len(RPs) > anti-affinity server group members, and then have to fit each member into a unique provider
15:55:04 mriedem but idk
15:55:12 mriedem sean-k-mooney: traits aren't the answer to everything :)
15:55:26 dansmith lol
15:55:36 mriedem when i give the project roadmap thing for R,
15:55:53 mriedem i think i'm going to say, "fix traits which has become the new bdm/quotas/local delete quagmire"
15:56:27 dansmith lol
15:56:31 mriedem roadmap for R: turn traits into CIMaaS
15:56:36 jaypipes mriedem: server groups are on my list of "figure out how to port this over to placement" things, yeah....
15:56:39 sean-k-mooney true i was thinking more of taging a resouce prvider in some way and saying only condider providers with this tag
15:56:55 jaypipes mriedem: for the immediate future, they will need to stay in the scheduler filters/weighers.
15:57:00 mriedem sean-k-mooney: that's more like an aggregate to me
15:57:06 jaypipes mriedem: right.
15:57:08 mriedem jaypipes: yeah that's fine
15:57:26 jaypipes mriedem: the problem with server groups has *always* been how to handle changes in the group over time.
15:57:43 sean-k-mooney mriedem: yes more of dynamcally created one though
15:58:07 mriedem sean-k-mooney: now you're just talking sci fi
15:58:13 mriedem :)
15:58:19 sean-k-mooney :)
15:58:30 mriedem don't forget we have to add support for migrating affinity groups as a unit
15:59:28 sean-k-mooney mriedem: that would be an entirely new feature right? currently you cant migrate a group just the instance of a group
16:00:04 sean-k-mooney that is going to create a lot of headaches if one of the member fails to migrate
16:00:14 mriedem sean-k-mooney: yes entirely new
16:00:29 mriedem sean-k-mooney: let me find some light reading for your evening
16:01:02 mriedem sean-k-mooney: https://review.openstack.org/#/c/440580/
16:01:35 mriedem basically we can spew out scheduler hints from the API so external systems can attempt to coordinate things, or we build something into nova
16:02:41 sean-k-mooney i see are the sceduler hints corrent stored in the db and this is just exposing them or are we also going start storeing them
16:02:50 sean-k-mooney just reading over it now
16:03:00 mriedem they are stored

Earlier   Later