Posted	Nick	Remark
#openstack-nova - 2019-10-15
20:36:50	efried	swhat I'm looking at now.
20:37:58	mriedem	in the related ML thread for that patch we talked about making the API only proceed if the service was down but nacked that idea for some reason
20:44:10	mriedem	fun it looks like my rechecks are being ignored
20:44:42	mriedem	dan rechecked https://review.opendev.org/#/c/634832/ hours ago with no results and it's not queued and i just rechecked it and it's still not queued
20:44:47	mriedem	fungi: ^
20:45:34	mriedem	my guess is because the comment doesn't start with "recheck", it starts with "(3 comments)"
20:45:50	mriedem	yup, now it's queued
20:45:54	mriedem	is that new behavior?
20:46:48	fungi	nope, it's just the way zuul is configured via a regular expression on the text of the comment event gerrit emits
20:47:15	mriedem	could have sworn i've issued rechecks while leaving comments before, but can't say for certain
20:47:34	fungi	if you leave a vote at the same time you add a recheck comment, it won't match the regex
20:48:16	mriedem	ok
20:48:19	fungi	https://opendev.org/openstack/project-config/src/branch/master/zuul.d/pipelines.yaml#L24
20:48:26	fungi	that's the current regex
20:48:45	fungi	i've noticed before that it ignores the recheck if i leave a vote with the same comment
20:49:34	fungi	i've never dug in with a sample comment event to see if that regex could be extended to accommodate it
20:50:00	mriedem	efried: are you working through bug 1841481 ?
20:50:00	openstack	bug 1841481 in OpenStack Compute (nova) "Race during ironic re-balance corrupts local RT ProviderTree and compute_nodes cache" [Medium,In progress] https://launchpad.net/bugs/1841481 - Assigned to Matt Riedemann (mriedem)
20:50:20	efried	mriedem: stale, but in my backlog
20:50:28	efried	is it time to get back to it?
20:50:35	mriedem	well, was just going to point out https://review.opendev.org/#/c/684840/ and above
20:50:37	efried	I feel guilty any time I do real code.
20:50:54	efried	there's always some f'in ptl thing that needs doing.
20:51:00	mriedem	your series deals with the corrupt provider tree cache, mine deals with the corrupt RT.compute_nodes cache
20:51:23	efried	so they need to be combined?
20:51:40	efried	or at least reconciled
20:53:01	mriedem	last i looked at yours i said,
20:53:02	mriedem	"Simply dealing with the ResourceTracker.compute_nodes invalid cache (issue #1 in the bug report) resolves the issue assuming the ProviderTree associations are considered stale. If the associations are stale in the ProviderTree cache, we likely still have a problem which is what Eric's series here is dealing with (but Eric's series doesn't deal with the ResourceTracker.compute_nodes aspect of the bug)."
20:54:25	mriedem	i'm not totally sure my functional recreate test hits all of the nuance with the provider tree cache but it does check _associations_stale
20:55:15	mriedem	the actual RT.compute_nodes cache fix is simple https://review.opendev.org/#/c/684849/2/nova/compute/resource_tracker.py
20:55:26	mriedem	don't cache the node unless RT._update is OK
21:04:05	mriedem	http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Skipping%20removal%20of%20allocations%20for%20deleted%20instances%3A%20Failed%20to%20retrieve%20allocations%20for%20resource%20provider%5C%22%20AND%20message%3A%5C%22No%20resource%20provider%20with%20uuid%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22%20AND%20project%3A%5C%22openstack%2Fironic%5C%22&from=7d
21:04:12	mriedem	we do see it in ironic multinode jobs
21:08:10	mriedem	maybe the functional test is too much in there, idk - could just do a simple unit test to make sure we don't save the node in RT.compute_nodes if _update fails like we did here https://review.opendev.org/#/c/675704/
21:14:12	openstackgerrit	Dan Smith proposed openstack/nova master: Add image caching API for aggregates https://review.opendev.org/687140
21:32:29	efried	clearly I'll need to set aside some time to swap that all in
21:32:36	efried	which isn't today unfortunately.
22:56:35	openstackgerrit	Merged openstack/nova master: Add cache_images() to conductor https://review.opendev.org/687139
22:56:51	openstackgerrit	Merged openstack/nova master: Fix legacy issues in filter migrations by user_id/project_id https://review.opendev.org/682198
23:10:45	openstackgerrit	Matt Riedemann proposed openstack/nova master: WIP: Add negative test to delete server during cross-cell resize claim https://review.opendev.org/688832
23:16:56	mriedem	drats, have to rebase the cross-cell resize series again
23:17:47	mriedem	dansmith: we can talk about it tomorrow or whenever but i wrote that negative test you asked for https://review.opendev.org/688832 and it exposes a latent bug in how MigrationTask.rollback works, which i think affects same-cell resize as well wrt leaked allocations on the source host
23:17:56	mriedem	but i'm basically done for the day as well
#openstack-nova - 2019-10-16
00:31:14	brinzhang	good morning, nova.
00:31:50	brinzhang	efried: good morning ^^
01:01:38	openstackgerrit	Brin Zhang proposed openstack/nova master: Add image caching API for aggregates https://review.opendev.org/687140
01:05:56	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.opendev.org/627890
01:05:56	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.opendev.org/634832
01:05:57	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.opendev.org/627891
01:05:57	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method https://review.opendev.org/635080
01:05:58	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.opendev.org/635646
01:05:58	openstackgerrit	Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.opendev.org/635668
01:05:59	openstackgerrit	Matt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize() https://review.opendev.org/635684
01:05:59	openstackgerrit	Matt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted() https://review.opendev.org/636224
01:06:00	openstackgerrit	Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.opendev.org/636253
01:06:01	openstackgerrit	Matt Riedemann proposed openstack/nova master: Handle target host cross-cell cold migration in conductor https://review.opendev.org/642591
01:06:01	openstackgerrit	Matt Riedemann proposed openstack/nova master: Validate image/create during cross-cell resize functional testing https://review.opendev.org/642592
01:06:02	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add zones wrinkle to TestMultiCellMigrate https://review.opendev.org/643450
01:06:02	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing https://review.opendev.org/643451
01:06:03	openstackgerrit	Matt Riedemann proposed openstack/nova master: Refresh instance in MigrationTask.execute Exception handler https://review.opendev.org/669012
01:06:03	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add negative test for prep_snapshot_based_resize_at_source failing https://review.opendev.org/669013
01:06:04	openstackgerrit	Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source compute method https://review.opendev.org/637058
01:06:04	openstackgerrit	Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.opendev.org/637070
01:06:05	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method https://review.opendev.org/637075
01:22:57	openstackgerrit	Matt Riedemann proposed openstack/nova master: Add image caching API for aggregates https://review.opendev.org/687140
02:11:39	openstackgerrit	zhangyujun proposed openstack/nova master: Should not raise when restore power on failed https://review.opendev.org/624854
02:21:21	openstackgerrit	Brin Zhang proposed openstack/nova master: Follow-up: Add delete_on_termination to volume-attach API https://review.opendev.org/680655
06:30:51	openstackgerrit	Merged openstack/nova master: Update compute rpc version alias for train https://review.opendev.org/688418
07:22:39	openstackgerrit	Brin Zhang proposed openstack/nova master: Follow-up: Add delete_on_termination to volume-attach API https://review.opendev.org/680655
07:27:42	gibi	good morning
07:47:11	openstackgerrit	Brin Zhang proposed openstack/python-novaclient master: Add minor version [21] to the test_versions https://review.opendev.org/688599
08:06:50	LiangFang	hi, where can I find the plan of Shanghai PTG? Thanks
08:07:02	LiangFang	e.g. the topics
08:07:18	LiangFang	how to add topic please?
08:15:41	melwitt	LiangFang: http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010126.html
08:18:39	kashyap	melwitt: Wauw, you're still awake?
08:20:08	LiangFang	melwitt: thanks
08:23:37	melwitt	kashyap: yeah :( going to bed soon
08:24:13	melwitt	thanks o/
08:43:58	frickler	KeithMnemonic: I haven't tested that scenario yet, but we're running the same setup, so I'd be interested in hearing more about your findings. If you discover a reproducible issue, it would probably be best to create a bug report on lp
09:13:12	openstackgerrit	sunjiyun-inspur proposed openstack/nova master: first modify https://review.opendev.org/688884
09:31:30	openstackgerrit	Huachang Wang proposed openstack/nova master: Set instance CPU policy to 'share' through image property https://review.opendev.org/688603
11:07:22	openstackgerrit	wangfaxin proposed openstack/nova master: Fix misspell word https://review.opendev.org/688901
12:53:04	openstackgerrit	Balazs Gibizer proposed openstack/nova master: Mask the token used to allow access to consoles https://review.opendev.org/220622
13:29:06	mriedem	bauzas: another scenario where we could leak allocations https://bugs.launchpad.net/nova/+bug/1848343
13:29:07	openstack	Launchpad bug 1848343 in OpenStack Compute (nova) "MigrationTask rollback can leak allocations for a deleted server" [Undecided,Triaged]
13:29:59	mriedem	does your audit command detect the scenario that a resource provider has VCPU allocations but the instance consumer does not exist?
13:30:43	dansmith	mriedem: guh, sorry about those last few issues, but thanks for fixing them
13:32:00	mriedem	np
13:32:31	mriedem	i reckon i'll spend some time today to write a recreate test for same-cell resize for that latent bug above
13:48:55	openstackgerrit	Huachang Wang proposed openstack/nova master: [WIP] To create single NUMA node instance in function '_get_numa_topology_auto' https://review.opendev.org/688932
13:48:55	openstackgerrit	Huachang Wang proposed openstack/nova master: [WIP] Assign and track instance pinning cpu through 'cpu_pinning' field https://review.opendev.org/688933
13:48:56	openstackgerrit	Huachang Wang proposed openstack/nova master: [WIP] Add a new instance CPU allocation policy: mixed https://review.opendev.org/688934
13:48:57	openstackgerrit	Huachang Wang proposed openstack/nova master: [WIP] virt/libvirt: Get host pin cpuset according instance cpu_pinning https://review.opendev.org/688935
13:48:57	openstackgerrit	Huachang Wang proposed openstack/nova master: [WIP] metadata: export the vCPU IDs that are pinning on the host CPUs https://review.opendev.org/688936
13:57:56	KeithMnemonic	frickler i will let you know what i find out. so far it seems some sort of access issue with the backend rbd disk during an evacuation
14:06:30	openstackgerrit	Dan Smith proposed openstack/python-novaclient master: Add aggregate-cache-images command and client routines https://review.opendev.org/687141
14:07:55	openstackgerrit	Matt Riedemann proposed openstack/python-novaclient master: Microversion 2.80: Add user_id/project_id to migration-list API https://review.opendev.org/675023
14:09:30	openstackgerrit	Matt Riedemann proposed openstack/python-novaclient master: Add aggregate-cache-images command and client routines https://review.opendev.org/687141
14:15:03	bauzas	mriedem: sorry, got some network issues last hour, just saw your ping

Earlier   Later