Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-16
21:20:26 efried Okay. I think we're going to go dumber than that. We're not CIing before newton, so we'll just say "if not newton". Thanks dansmith.
21:20:37 dansmith seems reasonable :)
21:32:09 edleafe bauzas: dansmith: I have another patch that adds returning multiple hosts to the conductor, but wasn't sure if there was an agreement on a) how to do it and b) how many
21:33:40 dansmith edleafe: should be num_retries or whatever
21:33:53 dansmith well, num_instances * num_retries
21:33:58 dansmith edleafe: there's another wrinkle
21:33:59 edleafe "whatever" :)
21:34:19 dansmith the hosts we slap into a reqspec to pass down to the cell all have to be ... in that cell
21:34:46 dansmith so I was thinking as a first go, we might just get the normal candidate hosts, and then for each one, ask for n-1 more that are in the same cell, since we can ask the scheduler that already
21:34:56 dansmith (for the same cell I mean)
21:35:19 edleafe so... the scheduler picks a host, and then only returns other hosts that are in the same cell?
21:35:25 edleafe IOW, a post-filter?
21:37:16 dansmith well, we pick num_instances hosts like normal,
21:37:35 dansmith but then for each instance we re-ask for num_retries host in the same cell
21:39:28 edleafe ok, so check the cell_uuid for each selected host, and only return more hosts with that same cell_uuid
21:40:06 dansmith yeah, we can already ask to limit hosts to a given cell_uuid,
21:40:32 dansmith so I guess we have to look up the host_mapping for that host so we know what cell_uuid to ask for,
21:40:49 dansmith but we have to do that somewhere in that code so we should be able to collapse those (eventually, if nothing else)
21:41:09 edleafe yeah, the order of events was not clear to me
21:41:17 dansmith but clear now?
21:41:37 openstackgerrit Sylvain Bauza proposed openstack/nova master: Scheduler returning more than 1 host https://review.openstack.org/465175
21:41:37 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: Allocate instance by the conductor https://review.openstack.org/460455
21:41:38 dansmith I haven't looked at bauzas' code from just now, it's possible he's doing some of that
21:41:52 bauzas just updated my change
21:42:19 bauzas edleafe: did that in https://review.openstack.org/465175 given the consensus we had
21:42:30 edleafe dansmith: clear except for the request for a particular cell
21:42:38 bauzas one of the reasons why it took me a while to upload the series
21:43:03 dansmith edleafe: there's code in the migrate path that demonstrates that, but reqspec.destination.cell can be set, which limits it to that cel
21:43:20 dansmith bauzas: okay cool
21:43:38 edleafe bauzas: ok, thanks. I was looking for getting the cell uuid from the reqspec
21:44:16 dansmith I need to run off for a bit, but I will look over bauzas' patches when I get back
21:44:33 edleafe And was it agreed to return a 2-tuple of (chosen_host, alternative_hosts), or simply a list of hosts, with the first one the chosen one?
21:45:22 bauzas edleafe: don't forget that the scheduler can return more than 1
21:45:35 dansmith edleafe: we can't really do it all in one query because of the cell restriction I think
21:45:37 bauzas hint : num_instances
21:46:40 dansmith anyway, biab
21:46:44 edleafe bauzas: yeah, of course I get that. We were originally returning a list of single hosts; we could now return a list of lists of hosts
21:46:47 dansmith don't kill each other while I'm gone
21:47:01 edleafe dansmith: you're no fun
21:47:09 bauzas it's 23:47
21:47:16 bauzas nothing good happens after midnight
21:47:17 dansmith okay fine, do and I'll congratulate the winner when I return
21:47:18 melwitt what bauzas has at https://review.openstack.org/#/c/465175/2/nova/scheduler/filter_scheduler.py@142 is what I was thinking, without doing a second query
21:47:43 melwitt that is, parse the list we get back the first time to pick alternatives in the same cell
21:48:05 dansmith oh because we're getting back more than subset size now
21:48:19 melwitt we always get the whole list I thought (all that pass all filters)
21:48:29 edleafe melwitt: that's correct
21:48:31 melwitt and then the first is picked unless subset size is bigger than 1
21:48:40 dansmith we return 10k hosts from scheduler to conductor?
21:48:54 edleafe dansmith: no
21:49:02 openstackgerrit Sylvain Bauza proposed openstack/nova master: Scheduler returning alternatives per destination https://review.openstack.org/465175
21:49:02 openstackgerrit Sylvain Bauza proposed openstack/nova master: WIP: Allocate instance by the conductor https://review.openstack.org/460455
21:49:10 edleafe we return 10K hosts from placement to scheduler (potentially)
21:49:14 bauzas I stop iterating over the list once we have enough alternatives
21:49:19 bauzas but please see $code
21:49:20 dansmith oh, I see that's happening in scheduler, yeah, okay
21:49:26 melwitt yeah
21:49:31 bauzas seriously, the big problem is not there
21:49:35 dansmith yeah I didn't think about doing that, but that's good
21:49:51 bauzas like I said yesterday in that channel, we stop using iterators
21:50:02 melwitt bauzas: yeah, that's exactly what I hoped to see, stop when have enough instead of doing a list comprehension over all
21:50:07 dansmith I guess I thought we weighted a subset of the hosts,
21:50:13 bauzas so in case we have 10K nodes, then we have a 10K list for filters and a 10K list for weighers
21:50:21 dansmith because on early scheduler runs, we'll be weighing 10k nodes just to pick one
21:50:28 bauzas instead of having iterators
21:50:42 bauzas so it took me 3 hours to see if I was able to quickly fix that
21:50:58 bauzas ie. using iterators for filters and weighers
21:51:08 bauzas looks to me it's possible, but I need more time
21:51:56 dansmith not sure how you can pick the highest score host without iterating all of them...
21:51:59 dansmith if that's what you mean
21:52:51 bauzas dansmith: I was thinking of adding a reservoir sampling method for that
21:52:52 edleafe filters have to iterate through all returned hosts. Weighers have to iterate through all filtered hosts
21:53:05 dansmith edleafe: right
21:53:14 bauzas we need to iterate for sure, but we don't need to put it all in memory
21:53:22 dansmith sure
21:53:35 dansmith anyway, I'll look more at this when I get back
21:54:24 bauzas so, for example, to-be-weighting hosts could be like having an index and a weight so we could just sort based on that
21:54:39 bauzas instead of sorting based on a full memory
21:54:53 bauzas and once we have filtered hosts, we don't really need to keep that in memory
21:55:26 bauzas since for the moment the scheduler can hardly support more than 400 nodes, I think it's not a problem bubbling up
21:55:42 bauzas but if we say for cells v2 that all the nodes will be verified, then I think we should fix that
21:56:11 bauzas and please, don't coufound iterators and lists
21:57:14 bauzas edleafe: oh, and FWIW, the 2-tuple (host, alts) is just for a priv method
21:57:23 bauzas edleafe: select_dests is returning a keyed dict
21:57:32 bauzas well, a list of keyed dicts rather
22:03:50 edleafe bauzas: yes, I do believe I've seen that code once or twice :)
22:04:32 edleafe I was just asking if that implementation was agreed upon at the summit
22:05:18 bauzas we didn't discussed on the interface
22:05:30 bauzas so you could be bikeshedding it :p
22:06:12 bauzas tbc, I think that since _schedule() is private, we shouldn't really care of what we return
22:06:22 edleafe nope, just trying to catch up beyond the etherpad https://etherpad.openstack.org/p/BOS-forum-move-claims-from-compute-to-scheduler
22:06:25 bauzas the crux should be around select_dests()
22:46:57 melwitt anyone else interested in preventing KeyErrors in exceptions in unit test (creates garbled text to stdout), please review dis: https://review.openstack.org/#/c/411615
23:00:41 melwitt woot, thanks
23:07:38 openstackgerrit melanie witt proposed openstack/nova master: WIP Cache database and message queue connection objects https://review.openstack.org/465042
#openstack-nova - 2017-05-17
00:33:56 openstackgerrit Artom Lifshitz proposed openstack/nova master: Use VIR_DOMAIN_BLOCK_REBASE_COPY_DEV when rebasing https://review.openstack.org/465205
00:55:22 armax mriedem: ping
00:55:29 mriedem armax: pong
00:55:37 armax quick question
00:55:55 armax since I know how much you love numa and sriov code in nova

Earlier   Later