Posted	Nick	Remark
#openstack-nova - 2019-04-16
15:18:57	dansmith	mriedem: that's a different thing
15:19:09	mriedem	sean-k-mooney: this has been this way for *years*
15:19:19	mriedem	danpb did all this work
15:19:22	dansmith	mriedem: the unknown image props don't get put into the object, so the compute nodes never see them, that's definitely true
15:19:45	sean-k-mooney	well the unkonw ones are for the scheduler only
15:20:25	mriedem	sean-k-mooney: https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_image_properties_isolation.py#L45 is an ImageMetaProps object,
15:20:35	mriedem	so if we don't know about the property, it won't be in that object and the filter can't filter on it
15:20:47	mriedem	https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_image_properties_isolation.py#L56
15:22:43	dansmith	same for image_props_filter
15:22:50	dansmith	I guess jsonfilter is only on host state
15:23:07	mriedem	yeah the jsonfilter is a whole other unvalidated piece of gorp
15:23:25	dansmith	yeah, but I thought it could operate on the image too, but no
15:23:54	NewBruce	heading off line for a while; bbl
15:24:14	dansmith	well, anyway, we definitely removed some functionality in the image props filter when we did that, but (a) it's been a long time with no real complaint that I know of and (b) I wouldn't call it an api breakage
15:25:21	mriedem	liberty https://review.openstack.org/#/c/76234/
15:25:36	dansmith	er, hmm, maybe we didn't actually
15:25:46	dansmith	I thought image props filter could do more generic matching, but it doesn't
15:26:02	mriedem	right it's like 3 or 4 properties
15:26:07	mriedem	hw version, type, arch something like that
15:26:09	mriedem	very specific
15:26:15	dansmith	which is why I was thinking about the jsonfilter
15:26:28	dansmith	so if there's not some other filter I'm thinking of I guess we're good
15:26:39	mriedem	it's AggregateImagePropertiesIsolation
15:26:46	mriedem	that's the generic one
15:26:49	sean-k-mooney	ok do we have teh same restion now with flavor extra specs
15:26:52	mriedem	to tie aggregates to images
15:27:29	dansmith	mriedem: ah, just a straight match to host meta I see
15:27:37	sean-k-mooney	yes
15:27:52	sean-k-mooney	this was used for think like runt this image on the NFV aggrate
15:27:56	dansmith	sean-k-mooney: no such restriction with flavors
15:29:22	dansmith	mriedem: mostly unrelated to this, I wanted to throw something out there just for maybe future use
15:29:30	mriedem	totally unrelated to this, but there are 2 simple changes below https://review.openstack.org/#/c/570202/ (which i need for cross-cell resize as well as rebuild from cell0) that have a +2 and i'm looking for another core to hit those
15:30:21	dansmith	mriedem: when I was reading the two numa specs today I was reminded of something I was thinking about earlier, related to cases where we have instances which store old-format data, like something stuck in their flavor which needs to be migrated
15:30:22	sean-k-mooney	dansmith: ok i had tought we still supported "bring your own" metadata key for image too i guess not
15:31:06	dansmith	mriedem: things that we would need to online_migrate or something, and things that we would be tempted to resolve with "meh, just migrate all your instances left one rack to clean those up".. kinda like the recent discussion about reshape for that stuff
15:31:30	dansmith	mriedem: we might benefit from a "needs upgrade" flag on the instance, that would be shown in detailed list, to admins only,
15:31:44	dansmith	where I could list all tenants and see instances that have "needs upgrade"
15:32:16	sean-k-mooney	so the old usecase woulould be achive with a member_of request in the flavor extra spec
15:32:25	dansmith	anything could set that flag, like compute manager when it notices some legacy data, or even libvirt when it notices something like a disk format that needs to be upgraded or something
15:32:51	mriedem	dansmith: if the thing setting the flag has to already calculate that it will set the flag, why not just do the online migration right then?
15:33:31	dansmith	mriedem: well, because things like the resource topology thing would need to be done to all instances on the compute node at the same time, and after a config change
15:33:45	dansmith	mriedem: referring to the thing stephenfin and jaypipes and bauzas and I were discussing last week
15:34:04	bauzas	FWIW, on a 1.5h meeting atm
15:34:15	bauzas	but listening
15:34:45	mriedem	so instances a,b,c need an upgrade, but the admin doesn't know how to upgrade them, i.e. is it migrating them, running the online_data_migrations cli, restarting the compute they are on, etc
15:35:22	mriedem	if the flag were an enum that's one way
15:35:41	dansmith	mriedem: yeah, so it'd be nice if we also tagged "issues" to the instance that you whittle down until the upgrade flag goes away, but I think that's too heavy for the moment.. but if we LOG.warning() for each instance that we were setting the flag on, then you would at least have a record
15:35:55	dansmith	mriedem: yeah, could be iterative, like service_version
15:36:01	dansmith	if we always did those things in order
15:36:22	sean-k-mooney	dansmith: could you tie it into a nova status check or something
15:36:42	mriedem	i don't think i'd rely on someone catching that warning
15:36:49	openstackgerrit	Merged openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises https://review.openstack.org/652146
15:36:50	mriedem	before their logs wrap or something
15:36:57	mriedem	depends on how long they retain stuff in es
15:37:08	mriedem	we already blast the logs with warnings that aren't useful
15:37:43	dansmith	mriedem: sure, it just makes it lighter for the first rev of the idea, but if we made it a "schema version" sort of iterative "level" like service_version that could be easy, as you would look up the reason in the decoder ring
15:38:21	cdent	Is there something up with the gate today? queue seems long and slow
15:38:22	dansmith	and we could translate it in the api to "is current or not" if the version is backlevel (and show the version I guess)
15:38:33	mriedem	cdent: it was like that yesterday too
15:38:53	mriedem	dansmith: i'm not sure what service_version you're referring to
15:38:59	mriedem	just the Service.version?
15:39:05	dansmith	no,
15:39:34	openstackgerrit	Merged openstack/nova stable/rocky: Don't warn on network-vif-unplugged event during live migration https://review.openstack.org/651797
15:39:37	dansmith	SERVICE_VERSION is a global counter of things we've done to services, so we can tell if they're up to date or not
15:40:23	dansmith	we could have a similar global "instance version" which was a little more like a schema version, where we record whether instance records had been modified for a specific transition
15:41:04	dansmith	this is not important right now, I was thinking you'd latch onto this so we could expose more info to the operators about upgraded-ness, but I'll just bring it up the next time it's appropriate, like in those specs
15:41:34	mriedem	ack
15:42:17	bauzas	sean-k-mooney: so, about the upgrade impact of the cpu-resources thing, I had an idea
15:42:38	bauzas	what we could do is potentially leave operators upgrading to Train without any impact
15:42:54	bauzas	ie. config options act exactly like Stein
15:43:02	bauzas	(for the existing ones)
15:43:07	mriedem	cdent: http://grafana.openstack.org/d/T6vSHcSik/zuul-status?panelId=3&fullscreen&orgId=1
15:43:27	bauzas	but, before you would like to use new config options, you'd have to say "hey, reshape my stuff"
15:43:29	sean-k-mooney	bauzas: yes. i suggested not enableing any of the new functionality if the new option was not set
15:43:31	bauzas	for this host
15:43:40	sean-k-mooney	yep
15:43:44	sean-k-mooney	so they would upgrade
15:43:49	bauzas	and in this case, we would model the new inventories
15:43:51	cdent	mriedem: "data without context can never be information"
15:44:01	sean-k-mooney	then move vms if need and then enable the new funcitonaltiy
15:44:10	bauzas	from an operator pov, it would mean "upgrade your cloud, upgrade your hosts"
15:44:14	mriedem	cdent: just saying there is a spike in the check queue so things are maybe slow as a result
15:44:14	sean-k-mooney	so they could to a rooling config update after the upgrade
15:44:21	bauzas	when you're done, then upgrade your config when you want
15:44:32	sean-k-mooney	yep
15:44:42	bauzas	a nova-status check could do the pre-flight check
15:44:45	sean-k-mooney	i think we are on the same page for that workflow
15:44:49	cdent	mriedem: sure, but we knew that already (or at least I did)
15:45:13	sean-k-mooney	bauzas: it requires use to support both codepaths in train
15:45:22	sean-k-mooney	then remove teh old one in U
15:45:29	bauzas	sean-k-mooney: yup, this was my concern
15:45:31	sean-k-mooney	but i think that will help with upgrades significantly
15:45:31	bauzas	but
15:45:37	mriedem	cdent: well you didn't tell me that, so who's misinformed now?!
15:45:52	bauzas	with addition to be : trigger a reshape by the operator
15:45:58	bauzas	or
15:46:05	bauzas	trigger a reshape by the config change
15:46:07	cdent	heh. touche
15:48:20	sean-k-mooney	bauzas: yep i was going to suggest that too.
15:48:52	sean-k-mooney	stephenfin: ^ when you get a chance after this call does the above make sense

Earlier   Later