Posted	Nick	Remark
#openstack-nova - 2018-03-21
22:35:22	mriedem	*RUN_ON_REBUILD
22:35:44	mriedem	also, remember the -5 to forcing a host during cold migration
22:35:57	cfriesen	mriedem: so why don't we just run all the filters and only evaluate the specified hostname?
22:36:30	cfriesen	other than "because that's how we've always done it"
22:36:45	mriedem	ask bauzas
22:37:04	mriedem	he talks at least semi-annually about changing the forst host/node stuff in server create to be a 'requested' destination
22:37:19	mriedem	evaluated by the scheduler
22:37:20	cfriesen	do it
22:37:21	cfriesen	;)
22:37:34	mriedem	that would be a microversion of course
22:37:52	mriedem	which i'd be ok with, it would be like passing a host to evacuate or live migrate
22:37:56	mriedem	and cold migrate now i guess
22:38:22	cfriesen	did we do a microversion when we started prefiltering via placement when forcing the destination?
22:39:09	cfriesen	I guess it's the claim in placement that I mean, not the prefiltering
22:41:42	mriedem	no, but i don't buy that argument
22:41:50	mriedem	i believe you could still fail the claim in the compute even if you force
22:42:27	mriedem	but i might be wrong, as i think the filters add the limits to the dict that gets passed down from scheduler to compute
22:43:10	mriedem	anyway, we have to claim in placement (Create the allocations) otherwise our tracking gets all screwed up, so i don't feel bad about breaking the 'force host' contract there
22:43:56	cfriesen	I think the limits happen after the filters...but in any case I get that we don't want to change behaviour without a microversion even if we never really defined the behaviour.
22:44:23	mriedem	the limits dict is passed from the scheduler down through conductor to the compute
22:44:25	mriedem	and used for the claim
22:44:34	mriedem	the limits dict in the scheduler is i believe populated via the filters
22:45:03	mriedem	which is why we can't remove the old school claim stuff in compute until we at least drop something like the caching scheduler, which doesn't use placement and relies on the late ass claim
22:46:49	cfriesen	I think you're right...I see the filters updating host_state.limits
22:46:56	mriedem	guh, i've triaged at least 2 bugs like this today https://bugs.launchpad.net/nova/+bug/1754543
22:46:57	openstack	Launchpad bug 1754543 in OpenStack Compute (nova) "not update request_spec.request_networks after attach or detach interface" [Undecided,Invalid] - Assigned to Deepak Mourya (mourya007)
22:47:06	mriedem	"request spec isn't what i expect it to be"
22:47:21	mriedem	how is that a bug, except for wanting to hack in your own private stuff
22:51:32	cfriesen	heh...so they want request spec to reflect the instance as it is now, rather than as it was at boot time?
22:51:53	sean-k-mooney[m]	mriedem: i havent read it yet but i would expect the request spec in the instance to be updated when i attach or detach an interface so livemigration would work right
22:52:23	mriedem	cfriesen: i guess
22:52:38	mriedem	sean-k-mooney[m]: we already have a thing that tracks that,
22:52:40	mriedem	it's called,
22:52:43	mriedem	the instance
22:53:06	mriedem	the request spec is not in the instance
22:53:16	mriedem	it's a copy of the initial server create request
22:53:19	cfriesen	mriedem: don't we feed the request spec to the scheduler when searching for a dest?
22:53:26	mriedem	minus ports and bdms
22:53:38	mriedem	cfriesen: sure do
22:53:42	mriedem	sometimes slightly modified
22:53:58	mriedem	so my guess would be, people have out of tree filters,
22:54:06	mriedem	and those filters need to know information about volumes and ports on the instance,
22:54:13	mriedem	and they want to get it from the request spec,
22:54:20	cfriesen	I was going to ask about ports/bdms, but then you pointed out that they were subtracted anyways.
22:54:23	mriedem	because they are too lazy to hit the DB, or cinder or neutron APIs
22:54:37	mriedem	they aren't subtracted, they just aren't persisted in the request spec
22:54:43	sean-k-mooney[m]	mriedem: well if we send the request spec to the schduler when livemigrating then would you not want it to have the list of networks you vm currently has instead of the inital set
22:55:08	mriedem	sean-k-mooney[m]: we don't have any filters that look at that
22:55:35	mriedem	"nova doesn't provide the thing my private out of tree filter needs" isn't a bug
22:56:14	sean-k-mooney[m]	the pci passthrough fileter looks at this
22:56:37	sean-k-mooney[m]	that said i guess we dont supprot hot attach for sriov interfaces currently
22:56:54	sean-k-mooney[m]	you can detach actully but just not attach
22:57:45	sean-k-mooney[m]	actully the pci passthroguh filter looks at the pci requests specs which i guess is a little different
22:57:50	mriedem	right
22:58:38	mriedem	now i'm sure it's possible to attach a port to an instance where the network that port is on is available to the current compute host, and then live migrate the instance to another host where that network is't available,
22:58:43	mriedem	but that's also a problem we have with server create today
22:59:28	sean-k-mooney[m]	i always forget how the routed networks stuff works but do they just track the subnets in placement or is there also a sechuler filter
22:59:42	mriedem	there isn't a filter
22:59:46	mriedem	they use aggregates somehow
22:59:52	mriedem	i still don't know how it actually works
23:00:00	sean-k-mooney[m]	mriedem: on boot that is "fine" because we can retry. on livemigrate not so much
23:00:02	mriedem	without doing the port stuff in conductor
23:03:01	sean-k-mooney[m]	well if the request spec is intended to store teh inital request then ya thats not a bug
23:07:15	sean-k-mooney[m]	mriedem: by the way was i chatting to you at the ptg about the pci white list parser bug i found. i think i tracked it to here https://github.com/openstack/nova/blob/master/nova/pci/whitelist.py#L58 do you know if there is a reason we use jsonutils directly here and not oslo.config
23:10:55	mriedem	sean-k-mooney[m]: wasn't me
23:12:06	sean-k-mooney[m]	not looking to blame :)
23:13:45	sean-k-mooney[m]	the call to jsonutils.loads(jsonspec) can raise error other then value error so if you have unicode in your whitelist the nova compute agent can die because the exception is not caught
23:14:05	mriedem	lyarwood was looking at something similar before the ptg
23:15:02	sean-k-mooney[m]	mriedem: i hard locked 15 server with this bug + a docker/centos kernel bug the week before the ptg
23:15:32	sean-k-mooney[m]	our lab time was interested in why our rack was suddenly draw 5% of the total phase
23:15:54	mriedem	efried: this is a fun one up your ksa alley https://bugs.launchpad.net/nova/+bug/1752152
23:15:56	openstack	Launchpad bug 1752152 in OpenStack Compute (nova) queens "Attach Volume Fails with secure call to cinder" [Undecided,Triaged]
23:17:33	sean-k-mooney[m]	any way i was talking to dug helmen about if i should "fix" it in oslo or nova ill proably submit a patch to both and see which one merges first
23:18:28	efried	mriedem: Looks like it has an owner?
23:20:48	mriedem	efried: dikonoor doesn't seem to actually be working on it
23:21:03	mriedem	maybe run that through the internal powervc sametime channel :)
23:21:14	mriedem	er verse
23:21:16	mriedem	er slack
23:21:35	efried	mriedem: Do we need to use https to get the version document??
23:22:17	mriedem	i assume https is what's in the service catalog?
23:23:13	mriedem	apparently you don't need a token to get the version document, which is ok with http
23:23:46	mriedem	so uh,
23:23:59	mriedem	how terrible would it be if we s/https/http/ in this cinderclient code?
23:24:20	efried	MITM?
23:24:29	mriedem	aaS
23:25:44	efried	mordred: Care to render an opinion? (TL;DR: is it okay to demote https to http if we're just getting the version document?)
23:26:15	cfriesen	any way to do a MiTM attack with the version document?
23:26:29	cfriesen	whoops, efried already said that
23:26:34	efried	Other than, like, corrupting it.
23:26:47	efried	Or maybe spoofing a microversion with a known security flaw?
23:26:54	cfriesen	yeah, that's what I was thinking
23:28:14	mriedem	otherwise i can probably hack something like where we actually create a cinderclient client object, and then use it's internal client to make the request
23:28:16	efried	mriedem: The alternative, though, is for this method (still in cinderclient) to use proper ksa loading instead of direct requests.get
23:28:18	mriedem	or, just use ksa
23:28:23	efried	yeah, any of that.
23:31:16	efried	mriedem: If we want to keep that method in cinderclient, we could add a kwarg that lets you pass in a ksa session. Then from nova use the _SESSION global which we've already loaded by that point.
23:31:17	openstackgerrit	Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081
23:31:25	efried	mriedem: Howzat sound?
23:31:37	mriedem	efried: i'd then have to plumb that through into cinderclient i think
23:31:44	efried	yes, that's what I mean.

Earlier   Later