Posted	Nick	Remark
#openstack-nova - 2018-12-11
21:33:59	mriedem	Dec 11 16:28:48.055772 ubuntu-xenial-rax-iad-0001127841 nova-compute[368]: DEBUG nova.compute.manager [req-4f071ef5-d1a5-4cfb-820b-2d562666e534 req-86897002-4096-4477-a6ae-19515f45c40f service nova] [instance: 01e6b7fe-bfa0-4d51-bcdf-03e65971ed74] Processing event network-vif-plugged-11c3d630-37f4-4fd3-ba88-8355f7253ea6 {{(pid=368) _process_instance_event /opt/stack/nova/nova/compute/manager.py:7952}}
21:34:11	mriedem	Dec 11 16:28:48.057381 ubuntu-xenial-rax-iad-0001127841 nova-compute[368]: DEBUG nova.compute.manager [req-4f071ef5-d1a5-4cfb-820b-2d562666e534 req-86897002-4096-4477-a6ae-19515f45c40f service nova] [instance: 01e6b7fe-bfa0-4d51-bcdf-03e65971ed74] Processing event network-vif-plugged-05bb2644-68b1-4f07-a44e-f2277b444792 {{(pid=368) _process_instance_event /opt/stack/nova/nova/compute/manager.py:7952}}
21:34:14	mriedem	looks like we got 6 of 7
21:34:45	mriedem	c1777114-78f7-4533-8725-d01139d29c02 was the missing port event
21:36:21	melwitt	that's a lot of ports
21:37:49	mriedem	not for cfriesen
21:37:55	melwitt	haha
21:38:03	mriedem	cfriesen eats that many ports for f'ing breakfast
21:38:43	cfriesen	Kellog's Port Loops...breakfast of champions.
21:42:59	openstackgerrit	Matt Riedemann proposed openstack/nova master: Remove GROUP BY clause from CellMapping.get_by_project_id https://review.openstack.org/619061
21:43:03	mriedem	zzzeek: ^ updated to remove that GROUP BY, i'm doing the grouping in python - thanks for the mysql mode note on that
21:46:05	melwitt	yup, only 6 network-vif-plugged events sent in the neutron service log matching 'network-vif-plugged', u'server_uuid': u'01e6b7fe-bfa0-4d51-bcdf-03e65971ed74
21:47:02	mriedem	http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22TaggedBootDevicesTest%5C%22%20AND%20message%3A%5C%22Timeout%20waiting%20for%20%5B('network-vif-plugged'%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d fwiw only hits on that one change
21:47:09	mriedem	so maybe it just got really really unlucky
21:47:20	melwitt	only event I find related to that port is:
21:47:22	melwitt	Dec 11 16:28:19.663912 ubuntu-xenial-rax-iad-0001127841 neutron-server[23986]: INFO neutron.notifiers.nova [-] Nova event response: {u'status': u'completed', u'tag': u'c1777114-78f7-4533-8725-d01139d29c02', u'name': u'network-changed', u'server_uuid': u'01e6b7fe-bfa0-4d51-bcdf-03e65971ed74', u'code': 200}
21:48:26	melwitt	I feel like I've seen that before, where network-changed can happen instead of network-vif-plugged. I can't remember why/how
21:51:43	mriedem	hmm, not sure, sean-k-mooney might remember the different cases in which those events are sent
21:52:20	mriedem	but neutron might be doing something like, oh we're already going to send the network-changed event for this port, so don't also send it for network-vif-plugged,
21:52:28	mriedem	maybe the events are queued up by port_id and it gets overwritten
21:52:52	melwitt	yeah, could be. I'll try to remember to pick sean-k-mooney's brain about it later
21:53:31	melwitt	thanks for the help in looking at it
22:10:32	openstackgerrit	Matt Riedemann proposed openstack/nova master: Lazy-load SchedulerReportClient from API and drop _create_client lock https://review.openstack.org/623246
22:10:32	openstackgerrit	Matt Riedemann proposed openstack/nova master: DRY up SchedulerReportClient init https://review.openstack.org/623247
22:10:35	mriedem	is jay done for the year?
22:11:30	codyhill	mriedem: Worked! I wish I could use Metadata service for this. But it looks like that is a bug between cloud-init and ironic. But with config-drive… This works for me! Thanks for the help!
22:11:48	mriedem	codyhill: cool
22:12:05	mriedem	maybe we should update the docs somewhere to mention that you need to set the config on nova-compute if you're using config drive...
22:12:57	mriedem	codyhill: any suggestions there ^ to help others from beating their heads?
22:49:36	openstackgerrit	Matt Riedemann proposed openstack/nova master: Cleanup vendordata docs https://review.openstack.org/624518
22:50:15	melwitt	mriedem: he's attending the oath techpulse conference this week, says penick
22:50:19	openstackgerrit	Matt Riedemann proposed openstack/nova master: Cleanup vendordata docs https://review.openstack.org/624518
22:50:20	mriedem	codyhill: ^
22:50:35	mriedem	oh great, techpulse
22:50:43	mriedem	we're flatlining here
22:54:07	melwitt	when I test fails for ssh timeout and says this:
22:54:09	melwitt	tempest.lib.exceptions.SSHTimeout: Connection to the 172.24.5.148 via SSH timed out.
22:54:09	melwitt	User: cirros, Password: O0*bEQwoQf2_bEu
22:54:45	melwitt	does that really mean it used that password to login to cirros? because that's not the usual password it says it will be "login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root."
22:57:36	melwitt	because that should never work I don't think
23:09:30	melwitt	oh, it doesn't use password "Creating ssh connection to '172.24.5.148:22' as 'cirros' with public key authentication"
23:11:20	mriedem	note infra is dealing with a cirros issue where ssh fails b/c networking isn't setup in the guest b/c config drive is too small
23:11:43	mriedem	https://bugs.launchpad.net/openstack-gate/+bug/1808010
23:11:43	openstack	Launchpad bug 1808010 in OpenStack-Gate "Tempest cirros boots fail due to lack of disk space" [Undecided,New]
23:12:24	melwitt	clicky
23:16:18	melwitt	looks like that's the culprit. I don't see the "no space left" messages but I do see "WARN: failed: route add -net "0.0.0.0/0" gw "10.1.0.1""
23:18:02	melwitt	so I guess it's not the same but has the same failure to add the route
23:35:32	melwitt	added a comment to lp
#openstack-nova - 2018-12-12
00:40:20	openstackgerrit	Merged openstack/nova master: Fix typo https://review.openstack.org/624442
00:52:25	mriedem	dansmith: i think i've figured out why we're getting that ocata api service version warning in the logs - https://review.openstack.org/#/c/624438/ - something is re-entering _get_instance with a cell1-targeted context, so it's failing to find any nova-osapi_compute services in the cell1 db (as expected), i'm just not sure where the re-entry is coming from.
00:56:20	mriedem	and that's why we don't get a 404 from the Instance.get_by_uuid call because the context is already targeted
01:04:21	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: add debug logging for bug 1807723 https://review.openstack.org/624181
01:04:21	openstack	bug 1807723 in tempest "test_volume_swap_with_multiattach intermittently fails during cleanup" [High,In progress] https://launchpad.net/bugs/1807723 - Assigned to Matt Riedemann (mriedem)
01:14:04	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: debug old compute API service version lookup https://review.openstack.org/624438
02:13:25	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
02:41:49	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
03:26:41	openstackgerrit	Yikun Jiang proposed openstack/nova master: Remove live_migration_progress_timeout config https://review.openstack.org/619142
03:35:45	openstackgerrit	Merged openstack/nova master: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/623282
03:58:19	openstackgerrit	Merged openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
07:09:10	openstackgerrit	Michael Still proposed openstack/nova master: Imagebackend should call processutils.execute directly. https://review.openstack.org/619704
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove final users of utils.execute() in libvirt. https://review.openstack.org/619705
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove the final user of utils.execute() from virt.images https://review.openstack.org/620007
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from the hyperv driver. https://review.openstack.org/620008
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from virt.disk.api. https://review.openstack.org/620009
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move bridge creation to privsep. https://review.openstack.org/620180
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move some linux network helpers to use privsep. https://review.openstack.org/621398
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move simple execute call to processutils. https://review.openstack.org/621527
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move interface enabling to privsep. https://review.openstack.org/621528
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move setting mac addresses for network devices to privsep. https://review.openstack.org/621529
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move interface disabling to privsep. https://review.openstack.org/622150
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move binding ips to privsep. https://review.openstack.org/622151
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: create_veth_pair is unused, remove it. https://review.openstack.org/624226
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: Create specialist set_macaddr_and_vlan helper. https://review.openstack.org/624227
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move create_tap_dev into privsep. https://review.openstack.org/624228
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move set_vf_interface_vlan to be with its only caller. https://review.openstack.org/624229
07:09:19	openstackgerrit	Michael Still proposed openstack/nova master: Move DHCP releasing to privsep. https://review.openstack.org/624230
09:10:01	openstackgerrit	Zhenyu Zheng proposed openstack/nova master: WIP run metadata api per cell https://review.openstack.org/624612
09:19:39	yan0s	hi all
09:20:04	yan0s	why isn't there a "nova quota-class-list" command?
09:20:25	yan0s	or "openstack quota list --class"
09:20:46	yan0s	how can I get list of defined quota classes?
09:51:37	ondrejme	yan0s: nova quota-defaults ?
10:40:37	openstackgerrit	Lee Yarwood proposed openstack/nova master: libvirt: Add workaround to cleanup instance dir during evac with rbd https://review.openstack.org/618478
11:34:19	yan0s	ondrejme: it is possible to set different quota per class
11:34:46	yan0s	and it possible to see the quota set in this class if you know the class name
11:35:09	yan0s	but it is not possible to get a list of available classes
11:36:07	yan0s	the command you sent me returns the default quota values
11:42:46	openstackgerrit	Lee Yarwood proposed openstack/nova stable/rocky: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/624677
11:53:08	stephenfin	lyarwood: I was wondering why nova-api (or was it the conductor?) was spewing all those logs ^
11:56:03	lyarwood	stephenfin: that's dansmith's fix, I'm just pulling it back into stable/rocky
12:12:58	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Sync STANDARDS with placement/nova rc_fields https://review.openstack.org/624370
12:13:00	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Tune up documentation to be more useful https://review.openstack.org/624384
12:56:23	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
12:58:29	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
13:39:54	adrianc	sean-k-mooney: Hi, regarding the issue you experienced yesterday with live-migration, ping me when you are here
13:46:23	sean-k-mooney	adrianc: ping
13:46:27	sean-k-mooney	:)
13:49:02	adrianc	sean-k-mooney: so apparently there is some weird behavior with sriov macvtap when you tear down the VM the VF netdev remains up
13:49:33	adrianc	so you have a scenario of duplicate macs in the network

Earlier   Later