Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-24
10:11:45 bauzas johnthetubaguy: so, AFAICU we're discussing on how to have a right interface
10:12:27 mdbooth We could add a new config variable which is a total, and immediately deprecate the old 2. The new value will default to the product of the deprecated values if not set explicitly. We hardcode 10 steps.
10:13:45 bauzas johnthetubaguy: so, you'd prefer to have https://review.openstack.org/#/c/417961/19/nova/scheduler/host_manager.py@635 in the scheduler report client ?
10:14:05 mdbooth johnthetubaguy: If this is up against the wire I can probably write that today myself, btw. Is this on anybody's radar?
10:14:18 johnthetubaguy mdbooth: we already have the total as a config setting I think, thats the delay between steps, not the size of the steps
10:14:37 johnthetubaguy mdbooth: pkoniszewski is usually responsive if he is around
10:14:42 mdbooth I don't think there's a total time, it's calculated.
10:15:08 pkoniszewski I'm around
10:15:16 pkoniszewski yeah, I also agree with mdbooth comment, very good point
10:15:41 johnthetubaguy bauzas: I think I would like to see the scheduler directly call the placement client, then pass the return value into the host_state to get back the list of objects
10:15:56 johnthetubaguy bauzas: it could go into the scheduler or the placement client I think
10:16:09 johnthetubaguy mdbooth: I think live_migration_downtime is the config I mean
10:16:15 bauzas johnthetubaguy: you mean the scheduler driver, not the host manager ?
10:16:27 bauzas johnthetubaguy: so we could just drill down a list of hosts to the host manager?
10:16:31 johnthetubaguy mdbooth: there is the total timeout, and the time to reach the total timeout in the steps
10:16:54 bauzas johnthetubaguy: I need to consider that if so
10:16:58 johnthetubaguy mdbooth: given we also define a time after which the live-migrate is considered failed and we give up, we could use that to define the steps
10:17:13 bauzas johnthetubaguy: I actually made that split because of the CachingScheduler, but lemme think about that loud
10:17:19 pkoniszewski johnthetubaguy: live migration timeout can be set to 0 to disable it
10:17:22 johnthetubaguy bauzas: I think we would pass resource providers into the host manager
10:17:30 pkoniszewski johnthetubaguy: however, nova still needs to increase downtime over time
10:17:33 johnthetubaguy bauzas: it shouldn't affect the caching scheduler
10:17:33 bauzas johnthetubaguy: I'm not sure I like that
10:17:43 bauzas johnthetubaguy: I would pass hosts, not RPs to the Hostmanager
10:17:45 gmann alex_xu: seems like we just broke the validation part, feature still should work
10:17:53 johnthetubaguy pkoniszewski: yeah, we would need a default if that timeout is zero
10:17:54 pkoniszewski we would need some kind of a magic number
10:18:03 bauzas johnthetubaguy: I mean, lemme clarify
10:18:25 bauzas johnthetubaguy: I think having the filter scheduler directly calling the reportclient to get the list of RPs is a good idea
10:18:29 alex_xu gmann: is there still a way to set tag to the bdm or network device?
10:18:45 johnthetubaguy pkoniszewski: it could be some multiple of the 500 seconds setting, if overall timeout is 0
10:18:47 bauzas johnthetubaguy: what I'm saying is that the translation from RP to compute nodes should still be filter scheduler thing
10:18:59 johnthetubaguy bauzas: agreed
10:19:14 johnthetubaguy bauzas: I think the filter scheduler method stays the same, its just whats in there that changes
10:19:15 bauzas johnthetubaguy: from an interface PoV, that's actually an excellent point
10:19:27 bauzas I mean, the filter scheduler returns compute nodes
10:19:34 johnthetubaguy bauzas: I kinda like passing the spec_obj into the placement client
10:19:34 bauzas the host manager returns host states
10:19:43 bauzas that, I'm not sure
10:20:06 johnthetubaguy ah, OK, you want that to be resource providers?
10:20:07 johnthetubaguy thats fair enough
10:20:15 bauzas johnthetubaguy: because I'd like to see the "spec to RC usage" logic be still a scheduler thing
10:20:38 bauzas the placement client shouldn't expect anything about how the scheduler works
10:21:03 mdbooth johnthetubaguy pkoniszewski: Sorry, even though I only read it yesterday, I still had to refresh it :)
10:21:05 gmann alex_xu: for bdm yes as there is no additionalProperties=False there but yes for network it is not allowed as additionalProperties=False :(
10:21:15 bauzas johnthetubaguy: in particular given I'm now very opiniated on having the resource query being optional depending on the filters
10:21:30 gmann alex_xu: i am modifying tempest device tag tests with all combination and with all version mentioned by you
10:21:39 mdbooth So there's live_migration_downtime, which is the maximum permitted downtime of an instance during live migration
10:21:41 gmann alex_xu: let's see how bdm tag behave on those
10:21:42 bauzas johnthetubaguy: the last 3 days convinced me we need to opt-in the filters
10:22:10 johnthetubaguy mdbooth: totally with you on need to refresh, I guess thats the problem the operators are having too
10:22:16 mdbooth Then there live_migration_downtime_steps, which is the number of steps to take towards that maximum, and live_migration_downtime_delay, which is the delay between each step.
10:22:17 bauzas johnthetubaguy: either way, thanks for reviewing, that's very good insight, and I can do a new iteration this morning
10:22:35 bauzas johnthetubaguy: FWIW, I'm stuck with grenade not liking the placement service in newton
10:22:49 johnthetubaguy bauzas: I was thinking it would ideally be an object we pass into the placement client (that could be empty)
10:22:51 mdbooth So the total time until we go with the maximum is live_migration_downtime_steps * live_migration_downtime_delay
10:22:58 mdbooth pkoniszewski: Am I reading that right? ^^^
10:22:59 alex_xu gmann: initial I also think there is no addtionalProperties, but I found it copy that from bdm v1 schema https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/block_device_mapping.py#L61
10:22:59 johnthetubaguy bauzas: ah yeah, that... :(
10:23:13 pkoniszewski johnthetubaguy: but im still considering a new config option, i see a case where operator might want to get to target downtime in a couple of seconds but still such live migration might need 10+ minutes to complete
10:23:16 bauzas johnthetubaguy: for the moment, I was seeing a dict
10:23:22 pkoniszewski mdbooth: right, that's the current state
10:23:27 bauzas johnthetubaguy: but longer term, why not an object, agreed
10:23:53 johnthetubaguy bauzas: honestly, a TODO in there saying this argument sucks might be all I need on that bit, lol
10:23:57 bauzas I'm just thinking it shouldn't be the Spec object
10:24:11 openstackgerrit Andrey Kurilin proposed openstack/python-novaclient: [CLI] Fix token auth type https://review.openstack.org/424256
10:24:12 johnthetubaguy bauzas: yeah, thats cool, I just re-read that file, it has to be resource providers
10:24:18 bauzas yup zactly
10:24:24 gmann alex_xu: ahh, yea, thanks for pointing that
10:24:42 bauzas and the scheduler has some internal logic for translating the Spec object into a ResourceProviderQuery object
10:24:48 mdbooth pkoniszewski: My idea just was to add a new option which is the total time, hardcode the number of steps to 10 as you've done, and default the total to the product of the set deprecated values (if they're set).
10:24:55 bauzas whatever the object is
10:24:58 johnthetubaguy bauzas: so maybe if you pass the return value of placement into host_manger get, even if thats just the list of uuids, that would be better
10:24:59 alex_xu gmann: np, I drop in that trap also :)
10:25:02 johnthetubaguy bauzas: yeah, that sounds good
10:25:12 johnthetubaguy bauzas: right now its a dict, but thats OK for now I guess
10:25:18 bauzas johnthetubaguy: I can forge a patch soon
10:25:29 mdbooth pkoniszewski: Incidentally, I also wonderered about setting guest.migrate_configure_max_downtime() on every iteration, regardless of steps.
10:25:35 mdbooth Would that be expensive?
10:25:36 bauzas johnthetubaguy: and I'll wait for sean about how to discuss the grenade thing
10:25:58 bauzas because the issue I guess is that we don't default running the placement service in newton
10:26:11 johnthetubaguy bauzas: cool, hopefully that helps, just didn't quite read as I expected
10:26:13 gmann alex_xu: heh
10:26:14 bauzas so, when upgrading, it fails because the config file is not correctly setup
10:26:42 bauzas that would suppose we would modify the config between newton and ocata, but that's pretty high impacty
10:27:01 johnthetubaguy bauzas: honestly, we might just need to support not having placement while the min_service_version is below a threshold, and only start using it after that point?
10:27:20 bauzas johnthetubaguy: that's why I'm waiting the US folks to be up before engaging that convo
10:27:22 bauzas :)
10:27:43 bauzas lots of discussions went already around about the upgrade path
10:27:50 johnthetubaguy mdbooth: we still need something that sets QEMU occasionally on some periodic task, but thats a good point, you can just work out where you should be on the line, and set that value
10:28:07 johnthetubaguy bauzas: yeah
10:28:22 johnthetubaguy bauzas: just curious why we don't do it the usual way, but thats for later
10:28:38 bauzas johnthetubaguy: meaning be gentle with not having the placement service for newton ?
10:28:58 bauzas and gracefully fallback if we don't have the version supported ?
10:29:23 bauzas that's one option but the consensus was that it was okay to have a maintenance window
10:29:51 johnthetubaguy bauzas: well its not really a maintenance window right, its just if you want no outage, you have to do more work
10:33:04 openstackgerrit Radomir Dopieralski proposed openstack/python-novaclient: ListExtResource given in place of ListExtManager https://review.openstack.org/424570
10:38:23 bauzas johnthetubaguy: tried to summarize our convo in https://review.openstack.org/#/c/417961/19/nova/scheduler/client/report.py@212, just drop a note if you think I misunderstood our agreement
10:38:34 gmann alex_xu: johnthetubaguy oomichi additionalProperties=False saved us many times but this we only got under its hammer :)
10:38:34 bauzas meanwhile, writing a new iteration

Earlier   Later