Posted	Nick	Remark
#openstack-nova - 2017-08-10
15:53:38	bauzas	I just don't want operators to freak out
15:53:39	dtantsur	well, in Queens the old style of scheduling is not going to be possible
15:53:46	dtantsur	we have to send some signal about that coming
15:54:00	bauzas	so it's just a wording issue
15:54:26	dtantsur	re upgrade https://docs.openstack.org/ironic/latest/admin/upgrade-guide.html#upgrading-from-ocata-to-pike
15:54:41	dtantsur	we should change s/recommended/required/, I'm going to have another update anyway
15:54:52	bauzas	but the point is, if you deprecate in Pike, that means you accept operators to not use those opts by the Pike timeframe, which could be a bit of concern if they roll out upgrades
15:55:06	bauzas	dtantsur: I briefly looked at your relnotes too
15:55:24	bauzas	dtantsur: and haven't found any clear ask for providing resource classes for nodes
15:55:37	dtantsur	bauzas: this is coming as part of https://review.openstack.org/#/c/491773/
15:55:37	bauzas	(talking of https://docs.openstack.org/releasenotes/ironic/unreleased.html )à
15:55:42	bauzas	ah cool
15:56:06	bauzas	dtantsur: so I guess some linkage between relnotes could be appreciated
15:56:21	bauzas	like, ironic has to be upgraded before compute nodes obviously
15:56:35	bauzas	but ironic can be upgraded after placement and scheduler, right
15:56:48	dtantsur	I guess so, yeah
15:57:14	bauzas	and before you run a fresh pike nova-compute, you have to update your ironic cloud to set the resource classes
15:57:29	bauzas	that is the upgrade ordering I care, since operators would query for
15:57:30	dtantsur	right, we can do it while still running Ocata
15:58:18	dtantsur	should this patch be finished today? I have meetings, then I'd prefer to bail out (I'm in EU time)
16:00:22	bauzas	dtantsur: you mean the notes ?
16:00:48	ildikov	mriedem: quick meeting if you're available
16:00:57	dtantsur	bauzas: this nova patch
16:01:10	bauzas	dtantsur: well, mriedem will cut the rc1 tag tonight for us (as well, I'm CEST) so that would mean those notes would require a backport if we want them in the pike tree
16:01:39	dtantsur	this sounds like "today"..
16:01:57	dtantsur	edleafe: is it possible you take on fixing release note wording for ^^^?
16:02:10	bauzas	dtantsur: ideally, I'd have appreciated to see https://review.openstack.org/#/c/491773/ landed first, but I guess we need to send them concurrently to the gate
16:03:04	dtantsur	bauzas: ironic is not branched today, so it may wait a bit, I think
16:03:14	edleafe	bauzas: dtantsur: I'll review after API-WG meeting
16:03:19	dtantsur	cool
16:03:30	bauzas	dtantsur: oh right, not the same cadence than us
16:03:51	bauzas	well, I can push a new rev
16:04:00	bauzas	edleafe: ^
16:04:01	dtantsur	if you don't mind!
16:04:16	edleafe	I never mind
16:04:20	edleafe	:)
16:06:03	mriedem	pushing what now?
16:06:13	mriedem	we already have stuff in the nova release notes that say ironic gets upgraded before nova
16:06:18	mriedem	for some unrelated features, like boot from volume
16:06:26	mriedem	we don't need to say that 5 times
16:07:10	bauzas	mriedem: context is https://review.openstack.org/#/c/492563/
16:10:12	edleafe	mriedem: it's the sixth time that will sink in
16:13:35	dtantsur	edleafe, bauzas: updated ironic part (docs and reno)
16:13:47	dtantsur	wording suggestions are welcome, my wording can be awful sometimes
16:14:07	mriedem	this doesn't have to be done in pike does it?
16:14:18	mriedem	seems like too many moving parts
16:14:21	dtantsur	mriedem: these filters won't work in Queens, no?
16:14:33	mriedem	idk
16:14:37	mriedem	i just,
16:14:58	mriedem	i can only have my head wrapped around 10 different RC1 stop ship omfg blocking issues at one time
16:15:00	mriedem	and this is #11
16:15:11	edleafe	unless we drag it out further, the plan is that in queens, ironic will only be scheduled using custom resource classes
16:15:42	edleafe	mriedem: this doesn't change functionality, right? Just marks things as deprecated
16:16:09	efried	If the placement API gives me a 500, and I'm using mod_wsgi, where do I find the log that's gonna tell me WHY? (devstack)
16:16:27	mriedem	efried: the placement-api logs
16:16:40	sdague	efried: uwsgi?
16:16:50	efried	mod-wsgi
16:16:57	efried	I know, I know, it's deprecated. Is that my actual problem?
16:17:00	sdague	ummm... I thought we deleted that bit
16:17:09	sdague	efried: mod-wsgi is going to be an apache log
16:17:15	efried	Heh. If it's deleted, not just deprecated, I guess that'd do it.
16:17:25	sdague	efried: we may not have deleted it
16:17:25	efried	Okay, where's that guy?
16:17:40	sdague	efried: whereever your apache logs
16:17:44	efried	I mean, our CI is running with it and (mostly) succeeding.
16:17:53	sdague	/var/log/apache/placement*
16:17:59	sdague	on an ubuntu like thing
16:18:17	sdague	it will be a different place on a redhat like thing
16:18:19	openstackgerrit	Merged openstack/nova master: remove log message with potential stale info https://review.openstack.org/492242
16:18:32	efried	nyaha! Thanks sdague
16:21:03	openstackgerrit	Sylvain Bauza proposed openstack/nova master: Deprecate bare metal filters https://review.openstack.org/492563
16:21:21	bauzas	dtantsur: edleafe: tried some rewording ^
16:21:31	dtantsur	thanks, looking
16:21:41	efried	That placement log has no entries since two weeks ago :(
16:21:53	efried	(and the 500 happened two minutes ago)
16:23:47	openstackgerrit	Merged openstack/nova master: placement: refactor healing of allocations in RT https://review.openstack.org/491850
16:26:02	sdague	efried: it is hard for me to debug remotely, if there is a box I can connect to I can look
16:26:18	sdague	I have IBM vpn access so hopefully a path
16:29:20	cdent	efried: there may still be a non wsgi log where the other nova logs are. another place to check are any other apache log you can find (grep for ‘resource_providers’). where things end up gets weird
16:29:38	cdent	and where a 500 ends up with mod_wsgi in the first place (even outside openstack) can be odd
16:29:50	cdent	sometimes it will be the central apache error.log
16:30:16	efried	cdent Yeah, I'm finding it in the horizon_access.log (which is odd - I'm not using horizon at all). But just the request/response headers, not the error trace.
16:30:41	cdent	is a horizon_error.log? or error.log?
16:31:17	efried	There's an error.log
16:31:35	efried	it has nothing for the last 6h
16:32:11	cdent	efried: I missed the earlier discussion, what time does this code come from?
16:32:40	efried	I'm using latest master nova. nova-powervm driver plugged in, but wouldn't think that's in the code path.
16:33:05	cdent	what installed placement?
16:33:43	sdague	cdent: I'm going to try to get on the box and poke to see if we can iterate through it
16:34:57	cdent	If it’s nova master there should still be a non apache log for errors, based on the work you did last summer sdague
16:35:21	cdent	_unless_ the 500 is in the calling of the wsgi app (rather than within the wsgi app)
16:35:36	cdent	I’d be curious to hear what it turns out to be
16:40:29	efried	esberglu We're still using mod-wsgi in our CI, right?
16:41:11	bauzas	folks, bailing out for a few hours, \o
16:41:49	efried	sdague cdent Ya know, my nova is current, but my other stacky stuff is a couple weeks old. Could that do it?
16:42:04	efried	I mean, regardless, I would like to be able to figure out where to look for the real cause of a generic 500...
16:43:49	cdent	efried: that’s why I asked about what’s doing your deployment (devstack, whatever)
16:44:50	efried	oh, sorry, I misunderstood the question then. Yeah, I did a devstack a couple weeks ago, been dinking around with the nova code since then. But hadn't yet had occasion to get this far (been just working conf stuff, so stopping early).
16:45:23	cdent	a devstack from two weeks ago that is still mod_wsgi surprises me
16:45:34	cdent	so it may be that your error is in journalctl

Earlier   Later