Posted	Nick	Remark
#openstack-nova - 2019-12-19
16:47:54	openstackgerrit	Stephen Finucane proposed openstack/nova stable/train: Block rebuild when NUMA topology changed https://review.opendev.org/698530
16:47:54	openstackgerrit	Stephen Finucane proposed openstack/nova stable/train: Disable NUMATopologyFilter on rebuild https://review.opendev.org/698532
17:45:21	openstackgerrit	Mykola Yakovliev proposed openstack/nova master: Fix boot_roles in InstanceSystemMetadata https://review.opendev.org/698040
18:33:54	openstackgerrit	Merged openstack/nova master: Fix race in test_create_servers_with_vpmem https://review.opendev.org/699997
19:22:27	gmann	stephenfin ping
20:09:22	openstackgerrit	Matt Riedemann proposed openstack/nova master: Ensure source service is up before resizing/migrating https://review.opendev.org/700062
20:25:18	openstackgerrit	Matt Riedemann proposed openstack/nova master: Ensure source service is up before resizing/migrating https://review.opendev.org/700062
20:55:29	mriedem	artom: stephenfin: since you summoned me https://review.opendev.org/#/c/698530/
21:02:13	mriedem	and https://review.opendev.org/#/c/698532/
21:02:35	mriedem	do we want to wait for https://review.opendev.org/#/c/698260/ to get cleaned up and backported on top of the train backports? because if so, someone is likely going to have to clean up https://review.opendev.org/#/c/698260/ for sean
21:14:28	openstackgerrit	Matt Riedemann proposed openstack/nova master: FUP for in-place numa rebuild https://review.opendev.org/698260
21:14:44	mriedem	done ^ let's get that approved on master and stacked on top of the backports
21:36:42	openstackgerrit	Matt Riedemann proposed openstack/nova stable/pike: pike-only: remove broken non-voting ceph jobs https://review.opendev.org/700072
21:45:15	TheJulia	Are there any known issues with nova-compute on the master branch right now?
21:45:37	TheJulia	like... ones that might show up with grenade running
21:46:09	mriedem	that's pretty broad
21:46:32	mriedem	you have a grenade job that rebalances ironic nodes between 2 computes right?
21:46:51	mriedem	there is a known race issue with that https://review.opendev.org/#/c/694802/
21:47:05	mriedem	https://bugs.launchpad.net/nova/+bug/1841481
21:47:05	openstack	Launchpad bug 1841481 in OpenStack Compute (nova) "Race during ironic re-balance corrupts local RT ProviderTree and compute_nodes cache" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)
21:49:23	TheJulia	mriedem: single node stack where nova-compute seems to die after being upgraded and started up for a little while
21:50:09	TheJulia	mriedem: but, I'm only asking broadly because I've only glanced at the logs enough to have a feeling it is nova-compute. I've not gone back and gone through everything comparing out the smoke test times yet
21:50:28	TheJulia	hence, why I asked if there is anything known before I go dig for a few hours
21:50:55	openstackgerrit	Matt Riedemann proposed openstack/nova stable/ocata: ocata-only: drop non-voting ceph job https://review.opendev.org/700077
21:51:02	mriedem	then not that i know of
21:51:29	TheJulia	mriedem: for context, two grenade jobs, one does a full upgrade, the other does a partial upgrade for historical reasons and that breaks most cycles leading to me wanting to drink.
21:51:38	TheJulia	Okay, yippie!
21:51:39	TheJulia	:)
22:26:57	TheJulia	hmm, in rescheduling for post upgrade smoke testing
#openstack-nova - 2019-12-20
01:12:27	openstackgerrit	Merged openstack/nova stable/stein: Zuul v3: use devstack-plugin-nfs-tempest-full https://review.opendev.org/699924
06:41:36	openstackgerrit	Merged openstack/nova master: Add functional cross-cell revert test with detached volume https://review.opendev.org/695335
06:41:43	openstackgerrit	Merged openstack/nova master: Refresh target cell instance after finish_snapshot_based_resize_at_dest https://review.opendev.org/698787
08:23:47	openstackgerrit	Luigi Toscano proposed openstack/nova stable/rocky: Zuul v3: use devstack-plugin-nfs-tempest-full https://review.opendev.org/700121
08:26:53	openstackgerrit	XiaojueGuan proposed openstack/nova master: Fix error out with an UnboundException when in build_and_run_instance https://review.opendev.org/698140
08:50:03	openstackgerrit	Brin Zhang proposed openstack/nova master: Add api for instance action details https://review.opendev.org/700123
09:04:45	openstackgerrit	XiaojueGuan proposed openstack/nova master: Fix error out with an UnboundException when in build_and_run_instance https://review.opendev.org/698140
09:05:00	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Add composable flavor properties https://review.opendev.org/663563
09:10:15	openstackgerrit	Chason Chan proposed openstack/nova master: Add description of live_migration_timeout_action option https://review.opendev.org/700124
09:42:43	openstackgerrit	Stephen Finucane proposed openstack/nova stable/train: FUP for in-place numa rebuild https://review.opendev.org/700127
10:50:02	openstackgerrit	Lee Yarwood proposed openstack/nova master: docs: Add note about an image signature validation limitation when using rbd https://review.opendev.org/700152
10:50:22	lyarwood	stephenfin: ^ simple doc wrinkle if you have time today
10:51:45	stephenfin	looks sane. +2
10:59:06	openstackgerrit	Stephen Finucane proposed openstack/nova master: FakeDriver: adding and removing instances on live migration. https://review.opendev.org/243613
11:41:02	openstackgerrit	Stephen Finucane proposed openstack/nova master: Print help if nova-manage subcommand is not specified https://review.opendev.org/694806
11:41:57	openstackgerrit	Merged openstack/nova master: FUP for in-place numa rebuild https://review.opendev.org/698260
11:47:48	openstackgerrit	Shilpa Devharakar proposed openstack/nova master: Handle new is_volume_backend join column query https://review.opendev.org/694462
11:47:48	openstackgerrit	Shilpa Devharakar proposed openstack/nova master: Instance object changes for the new 'is_volume_backed' expected_attr https://review.opendev.org/694463
11:47:49	openstackgerrit	Shilpa Devharakar proposed openstack/nova master: Ignore root_gb if instance is booted from volume https://review.opendev.org/612626
12:09:07	openstackgerrit	Stephen Finucane proposed openstack/nova stable/train: FUP for in-place numa rebuild https://review.opendev.org/700127
12:10:54	stephenfin	lyarwood: fixed https://review.opendev.org/#/c/700127/
12:15:35	lyarwood	stephenfin: ack, grabbing lunch and then I'll queue that series up
12:28:17	stephenfin	bauzas: You bored? I have a load of patches with single +2 that could do with a little push
12:28:40	stephenfin	And I think you're the only other core still around. Lucky you ;)
13:59:56	bauzas	stephenfin: heh, sure
14:00:04	bauzas	stephenfin: sorry, I was on gym :)
14:09:13	mriedem	lyarwood: i'm +2 on these train numa rebuild backports now if you want to send them on https://review.opendev.org/#/q/topic:bug/1763766+status:open
14:10:01	lyarwood	mriedem: ack, have them open now, looking.
14:11:36	TheJulia	Can anyone provide background on the "forbidden trait COMPUTE_STATUS_DISABLED" ?
14:12:21	mriedem	TheJulia: https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#compute-disabled-status-support
14:14:50	TheJulia	hmm... then the question becomes why does nova think the nova-compute for ironic is no longer working when it just deleted an instance :\
14:15:04	TheJulia	well, minutes before in the same running process :\
14:15:10	TheJulia	logs
14:18:28	TheJulia	mriedem: is it the conductor that is making that declaration ?
14:20:58	mriedem	no,
14:21:22	mriedem	if the compute service is enabled/disabled in the rest api then the trait added to or removed from the associated nodes that service is managing
14:21:37	mriedem	some virt drivers (i think only libvirt) will auto-enable/disable if the hypervisor is down/up
14:21:48	mriedem	so it could also be the compute via the driver that sets it
14:22:14	mriedem	you can disable a compute service and still delete instances from it
14:22:17	TheJulia	I was thought that there was some automagical thing close to the compute driver... sifting through the n-cpu logs very carefully at the moment
14:22:20	mriedem	disable != down
14:22:29	TheJulia	disable from scheduling, that makes sense
14:22:38	mriedem	correct
14:27:39	mriedem	are you seeing the logging from here? https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L546-L553
14:28:44	TheJulia	mriedem: nope :\
14:29:41	mriedem	does the ironic grenade multinode job disable one compute before upgrading it which triggers a rebalance of nodes from one compute to another?
14:31:29	TheJulia	this is a single stack job so everything get stopped and restarted
14:32:30	TheJulia	I see a get to os-service early on, but no puts
14:32:32	TheJulia	err, posts
14:32:33	mriedem	and the ironic virt driver doesn't auto enable/disable the compute service does it?
14:32:41	mriedem	the disable/enable is a PUT
14:33:02	TheJulia	oh, good, then none at all in the nova api log :)
14:33:07	mriedem	https://docs.openstack.org/api-ref/compute/#compute-services-os-services
14:33:29	TheJulia	I'm honestly don't remember if we auto-disable or not. I guess that is the next thing to look at
14:34:59	mriedem	i don't see anything in the driver
15:06:59	openstackgerrit	Matt Riedemann proposed openstack/nova master: Handle cell failures in get_compute_nodes_by_host_or_node https://review.opendev.org/700186
15:54:51	bauzas	stephenfin: don't hesitate to ping me the changes you'd like me to review
15:55:09	TheJulia	mriedem: by chance is that last patch related to our discussion? :)
15:55:56	mriedem	TheJulia: nope
15:56:15	mriedem	bauzas: bottom 4 nova-net removal changes are +2ed and ready to go https://review.opendev.org/#/c/696510/
15:56:18	mriedem	starting at ^
15:56:31	bauzas	mriedem: thanks !
15:56:42	bauzas	I'm finally back upstream this week FWIW
16:00:08	TheJulia	mriedem: rats, I was hopeful :)
16:01:14	openstackgerrit	Lee Yarwood proposed openstack/nova master: guestfs: Log a useful error when the required vmlinuz isn't found https://review.opendev.org/700193
16:10:37	openstackgerrit	Merged openstack/nova stable/train: Remove 'test_cold_migrate_with_physnet_fails' test https://review.opendev.org/700021
16:12:54	mriedem	gibi_pto: nits in https://review.opendev.org/#/c/699015/ for when you want to start a FUP patch for that series
16:16:45	mriedem	bauzas: also, the bottom few changes in the live migration + qos ports series ^ are +2ed and need another core
16:17:08	mriedem	per usual gibi has done a lot of good functional testing
16:33:02	bauzas	mriedem: ack, will review it too on monday
16:36:54	mriedem	johnthetubaguy: are you ok with me fixing https://review.opendev.org/#/c/697162/7/nova/tests/unit/compute/test_shelve.py@168 in a follow up?

Earlier   Later