Posted	Nick	Remark
#openstack-nova - 2018-12-11
21:43:03	mriedem	zzzeek: ^ updated to remove that GROUP BY, i'm doing the grouping in python - thanks for the mysql mode note on that
21:46:05	melwitt	yup, only 6 network-vif-plugged events sent in the neutron service log matching 'network-vif-plugged', u'server_uuid': u'01e6b7fe-bfa0-4d51-bcdf-03e65971ed74
21:47:02	mriedem	http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22TaggedBootDevicesTest%5C%22%20AND%20message%3A%5C%22Timeout%20waiting%20for%20%5B('network-vif-plugged'%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d fwiw only hits on that one change
21:47:09	mriedem	so maybe it just got really really unlucky
21:47:20	melwitt	only event I find related to that port is:
21:47:22	melwitt	Dec 11 16:28:19.663912 ubuntu-xenial-rax-iad-0001127841 neutron-server[23986]: INFO neutron.notifiers.nova [-] Nova event response: {u'status': u'completed', u'tag': u'c1777114-78f7-4533-8725-d01139d29c02', u'name': u'network-changed', u'server_uuid': u'01e6b7fe-bfa0-4d51-bcdf-03e65971ed74', u'code': 200}
21:48:26	melwitt	I feel like I've seen that before, where network-changed can happen instead of network-vif-plugged. I can't remember why/how
21:51:43	mriedem	hmm, not sure, sean-k-mooney might remember the different cases in which those events are sent
21:52:20	mriedem	but neutron might be doing something like, oh we're already going to send the network-changed event for this port, so don't also send it for network-vif-plugged,
21:52:28	mriedem	maybe the events are queued up by port_id and it gets overwritten
21:52:52	melwitt	yeah, could be. I'll try to remember to pick sean-k-mooney's brain about it later
21:53:31	melwitt	thanks for the help in looking at it
22:10:32	openstackgerrit	Matt Riedemann proposed openstack/nova master: DRY up SchedulerReportClient init https://review.openstack.org/623247
22:10:32	openstackgerrit	Matt Riedemann proposed openstack/nova master: Lazy-load SchedulerReportClient from API and drop _create_client lock https://review.openstack.org/623246
22:10:35	mriedem	is jay done for the year?
22:11:30	codyhill	mriedem: Worked! I wish I could use Metadata service for this. But it looks like that is a bug between cloud-init and ironic. But with config-drive… This works for me! Thanks for the help!
22:11:48	mriedem	codyhill: cool
22:12:05	mriedem	maybe we should update the docs somewhere to mention that you need to set the config on nova-compute if you're using config drive...
22:12:57	mriedem	codyhill: any suggestions there ^ to help others from beating their heads?
22:49:36	openstackgerrit	Matt Riedemann proposed openstack/nova master: Cleanup vendordata docs https://review.openstack.org/624518
22:50:15	melwitt	mriedem: he's attending the oath techpulse conference this week, says penick
22:50:19	openstackgerrit	Matt Riedemann proposed openstack/nova master: Cleanup vendordata docs https://review.openstack.org/624518
22:50:20	mriedem	codyhill: ^
22:50:35	mriedem	oh great, techpulse
22:50:43	mriedem	we're flatlining here
22:54:07	melwitt	when I test fails for ssh timeout and says this:
22:54:09	melwitt	User: cirros, Password: O0*bEQwoQf2_bEu
22:54:09	melwitt	tempest.lib.exceptions.SSHTimeout: Connection to the 172.24.5.148 via SSH timed out.
22:54:45	melwitt	does that really mean it used that password to login to cirros? because that's not the usual password it says it will be "login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root."
22:57:36	melwitt	because that should never work I don't think
23:09:30	melwitt	oh, it doesn't use password "Creating ssh connection to '172.24.5.148:22' as 'cirros' with public key authentication"
23:11:20	mriedem	note infra is dealing with a cirros issue where ssh fails b/c networking isn't setup in the guest b/c config drive is too small
23:11:43	openstack	Launchpad bug 1808010 in OpenStack-Gate "Tempest cirros boots fail due to lack of disk space" [Undecided,New]
23:11:43	mriedem	https://bugs.launchpad.net/openstack-gate/+bug/1808010
23:12:24	melwitt	clicky
23:16:18	melwitt	looks like that's the culprit. I don't see the "no space left" messages but I do see "WARN: failed: route add -net "0.0.0.0/0" gw "10.1.0.1""
23:18:02	melwitt	so I guess it's not the same but has the same failure to add the route
23:35:32	melwitt	added a comment to lp
#openstack-nova - 2018-12-12
00:40:20	openstackgerrit	Merged openstack/nova master: Fix typo https://review.openstack.org/624442
00:52:25	mriedem	dansmith: i think i've figured out why we're getting that ocata api service version warning in the logs - https://review.openstack.org/#/c/624438/ - something is re-entering _get_instance with a cell1-targeted context, so it's failing to find any nova-osapi_compute services in the cell1 db (as expected), i'm just not sure where the re-entry is coming from.
00:56:20	mriedem	and that's why we don't get a 404 from the Instance.get_by_uuid call because the context is already targeted
01:04:21	openstack	bug 1807723 in tempest "test_volume_swap_with_multiattach intermittently fails during cleanup" [High,In progress] https://launchpad.net/bugs/1807723 - Assigned to Matt Riedemann (mriedem)
01:04:21	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: add debug logging for bug 1807723 https://review.openstack.org/624181
01:14:04	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: debug old compute API service version lookup https://review.openstack.org/624438
02:13:25	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
02:41:49	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
03:26:41	openstackgerrit	Yikun Jiang proposed openstack/nova master: Remove live_migration_progress_timeout config https://review.openstack.org/619142
03:35:45	openstackgerrit	Merged openstack/nova master: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/623282
03:58:19	openstackgerrit	Merged openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
07:09:10	openstackgerrit	Michael Still proposed openstack/nova master: Imagebackend should call processutils.execute directly. https://review.openstack.org/619704
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove the final user of utils.execute() from virt.images https://review.openstack.org/620007
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove final users of utils.execute() in libvirt. https://review.openstack.org/619705
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from virt.disk.api. https://review.openstack.org/620009
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from the hyperv driver. https://review.openstack.org/620008
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move bridge creation to privsep. https://review.openstack.org/620180
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move simple execute call to processutils. https://review.openstack.org/621527
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move some linux network helpers to use privsep. https://review.openstack.org/621398
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move setting mac addresses for network devices to privsep. https://review.openstack.org/621529
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move interface enabling to privsep. https://review.openstack.org/621528
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move binding ips to privsep. https://review.openstack.org/622151
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move interface disabling to privsep. https://review.openstack.org/622150
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: Create specialist set_macaddr_and_vlan helper. https://review.openstack.org/624227
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: create_veth_pair is unused, remove it. https://review.openstack.org/624226
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move set_vf_interface_vlan to be with its only caller. https://review.openstack.org/624229
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move create_tap_dev into privsep. https://review.openstack.org/624228
07:09:19	openstackgerrit	Michael Still proposed openstack/nova master: Move DHCP releasing to privsep. https://review.openstack.org/624230
09:10:01	openstackgerrit	Zhenyu Zheng proposed openstack/nova master: WIP run metadata api per cell https://review.openstack.org/624612
09:19:39	yan0s	hi all
09:20:04	yan0s	why isn't there a "nova quota-class-list" command?
09:20:25	yan0s	or "openstack quota list --class"
09:20:46	yan0s	how can I get list of defined quota classes?
09:51:37	ondrejme	yan0s: nova quota-defaults ?
10:40:37	openstackgerrit	Lee Yarwood proposed openstack/nova master: libvirt: Add workaround to cleanup instance dir during evac with rbd https://review.openstack.org/618478
11:34:19	yan0s	ondrejme: it is possible to set different quota per class
11:34:46	yan0s	and it possible to see the quota set in this class if you know the class name
11:35:09	yan0s	but it is not possible to get a list of available classes
11:36:07	yan0s	the command you sent me returns the default quota values
11:42:46	openstackgerrit	Lee Yarwood proposed openstack/nova stable/rocky: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/624677
11:53:08	stephenfin	lyarwood: I was wondering why nova-api (or was it the conductor?) was spewing all those logs ^
11:56:03	lyarwood	stephenfin: that's dansmith's fix, I'm just pulling it back into stable/rocky
12:12:58	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Sync STANDARDS with placement/nova rc_fields https://review.openstack.org/624370
12:13:00	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Tune up documentation to be more useful https://review.openstack.org/624384
12:56:23	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
12:58:29	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
13:39:54	adrianc	sean-k-mooney: Hi, regarding the issue you experienced yesterday with live-migration, ping me when you are here
13:46:23	sean-k-mooney	adrianc: ping
13:46:27	sean-k-mooney	:)
13:49:02	adrianc	sean-k-mooney: so apparently there is some weird behavior with sriov macvtap when you tear down the VM the VF netdev remains up
13:49:33	adrianc	so you have a scenario of duplicate macs in the network
13:50:22	adrianc	doing ifconfig <VF-netdev> down on the source VF should fix the issue (at least on my setup it did)
13:52:38	adrianc	sean-k-mooney: i had a discussion with moshele about it, apparently this is an old issue which branches off this mail: https://www.redhat.com/archives/libvir-list/2016-September/msg00075.html
13:53:13	sean-k-mooney	ah ok
13:53:32	sean-k-mooney	so i had considered creating an os-vif plugin for sriov this cycle.
13:53:38	adrianc	sriov netdev mac address will be re-allocated only on driver rebind on the VF
13:54:02	sean-k-mooney	we can do it in the nova tree but we could also jsut create the plugin and have it set the vf down
13:54:26	adrianc	if you do passthrough then it will be properly cleaned, however on macvtap this is not the case
13:54:36	sean-k-mooney	ya
13:54:51	adrianc	i added a hook in my code to do: interface down in vif.py under unplug_hw_veb()
13:55:14	sean-k-mooney	ok cool that should work for now.

Earlier   Later