Posted	Nick	Remark
#openstack-nova - 2018-12-12
00:52:25	mriedem	dansmith: i think i've figured out why we're getting that ocata api service version warning in the logs - https://review.openstack.org/#/c/624438/ - something is re-entering _get_instance with a cell1-targeted context, so it's failing to find any nova-osapi_compute services in the cell1 db (as expected), i'm just not sure where the re-entry is coming from.
00:56:20	mriedem	and that's why we don't get a 404 from the Instance.get_by_uuid call because the context is already targeted
01:04:21	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: add debug logging for bug 1807723 https://review.openstack.org/624181
01:04:21	openstack	bug 1807723 in tempest "test_volume_swap_with_multiattach intermittently fails during cleanup" [High,In progress] https://launchpad.net/bugs/1807723 - Assigned to Matt Riedemann (mriedem)
01:14:04	openstackgerrit	Matt Riedemann proposed openstack/nova master: DNM: debug old compute API service version lookup https://review.openstack.org/624438
02:13:25	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
02:41:49	openstackgerrit	Yikun Jiang proposed openstack/nova master: Add live migration timeout action https://review.openstack.org/619143
03:26:41	openstackgerrit	Yikun Jiang proposed openstack/nova master: Remove live_migration_progress_timeout config https://review.openstack.org/619142
03:35:45	openstackgerrit	Merged openstack/nova master: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/623282
03:58:19	openstackgerrit	Merged openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
07:09:10	openstackgerrit	Michael Still proposed openstack/nova master: Imagebackend should call processutils.execute directly. https://review.openstack.org/619704
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove final users of utils.execute() in libvirt. https://review.openstack.org/619705
07:09:11	openstackgerrit	Michael Still proposed openstack/nova master: Remove the final user of utils.execute() from virt.images https://review.openstack.org/620007
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from the hyperv driver. https://review.openstack.org/620008
07:09:12	openstackgerrit	Michael Still proposed openstack/nova master: Remove utils.execute() from virt.disk.api. https://review.openstack.org/620009
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010
07:09:13	openstackgerrit	Michael Still proposed openstack/nova master: Move bridge creation to privsep. https://review.openstack.org/620180
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move some linux network helpers to use privsep. https://review.openstack.org/621398
07:09:14	openstackgerrit	Michael Still proposed openstack/nova master: Move simple execute call to processutils. https://review.openstack.org/621527
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move interface enabling to privsep. https://review.openstack.org/621528
07:09:15	openstackgerrit	Michael Still proposed openstack/nova master: Move setting mac addresses for network devices to privsep. https://review.openstack.org/621529
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move interface disabling to privsep. https://review.openstack.org/622150
07:09:16	openstackgerrit	Michael Still proposed openstack/nova master: Move binding ips to privsep. https://review.openstack.org/622151
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: create_veth_pair is unused, remove it. https://review.openstack.org/624226
07:09:17	openstackgerrit	Michael Still proposed openstack/nova master: Create specialist set_macaddr_and_vlan helper. https://review.openstack.org/624227
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move create_tap_dev into privsep. https://review.openstack.org/624228
07:09:18	openstackgerrit	Michael Still proposed openstack/nova master: Move set_vf_interface_vlan to be with its only caller. https://review.openstack.org/624229
07:09:19	openstackgerrit	Michael Still proposed openstack/nova master: Move DHCP releasing to privsep. https://review.openstack.org/624230
09:10:01	openstackgerrit	Zhenyu Zheng proposed openstack/nova master: WIP run metadata api per cell https://review.openstack.org/624612
09:19:39	yan0s	hi all
09:20:04	yan0s	why isn't there a "nova quota-class-list" command?
09:20:25	yan0s	or "openstack quota list --class"
09:20:46	yan0s	how can I get list of defined quota classes?
09:51:37	ondrejme	yan0s: nova quota-defaults ?
10:40:37	openstackgerrit	Lee Yarwood proposed openstack/nova master: libvirt: Add workaround to cleanup instance dir during evac with rbd https://review.openstack.org/618478
11:34:19	yan0s	ondrejme: it is possible to set different quota per class
11:34:46	yan0s	and it possible to see the quota set in this class if you know the class name
11:35:09	yan0s	but it is not possible to get a list of available classes
11:36:07	yan0s	the command you sent me returns the default quota values
11:42:46	openstackgerrit	Lee Yarwood proposed openstack/nova stable/rocky: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/624677
11:53:08	stephenfin	lyarwood: I was wondering why nova-api (or was it the conductor?) was spewing all those logs ^
11:56:03	lyarwood	stephenfin: that's dansmith's fix, I'm just pulling it back into stable/rocky
12:12:58	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Sync STANDARDS with placement/nova rc_fields https://review.openstack.org/624370
12:13:00	openstackgerrit	Chris Dent proposed openstack/os-resource-classes master: Tune up documentation to be more useful https://review.openstack.org/624384
12:56:23	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
12:58:29	openstackgerrit	Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689
13:39:54	adrianc	sean-k-mooney: Hi, regarding the issue you experienced yesterday with live-migration, ping me when you are here
13:46:23	sean-k-mooney	adrianc: ping
13:46:27	sean-k-mooney	:)
13:49:02	adrianc	sean-k-mooney: so apparently there is some weird behavior with sriov macvtap when you tear down the VM the VF netdev remains up
13:49:33	adrianc	so you have a scenario of duplicate macs in the network
13:50:22	adrianc	doing ifconfig <VF-netdev> down on the source VF should fix the issue (at least on my setup it did)
13:52:38	adrianc	sean-k-mooney: i had a discussion with moshele about it, apparently this is an old issue which branches off this mail: https://www.redhat.com/archives/libvir-list/2016-September/msg00075.html
13:53:13	sean-k-mooney	ah ok
13:53:32	sean-k-mooney	so i had considered creating an os-vif plugin for sriov this cycle.
13:53:38	adrianc	sriov netdev mac address will be re-allocated only on driver rebind on the VF
13:54:02	sean-k-mooney	we can do it in the nova tree but we could also jsut create the plugin and have it set the vf down
13:54:26	adrianc	if you do passthrough then it will be properly cleaned, however on macvtap this is not the case
13:54:36	sean-k-mooney	ya
13:54:51	adrianc	i added a hook in my code to do: interface down in vif.py under unplug_hw_veb()
13:55:14	sean-k-mooney	ok cool that should work for now.
13:56:02	sean-k-mooney	i would like to move that into os-vif later i also want to move the trusted vf logic into os-vif which was why i was orignally gong to create the sriov plugin
13:56:25	sean-k-mooney	adrianc: do you have a new patchset up with that change. ill redeploy and test
13:58:34	adrianc	not yet, i need to deploy the change and make sure it actually does the job. it worked manually.
14:00:30	adrianc	sean-k-mooney: it should not be tied to the neutron-sriov-live-migration work as it serves as a WA to an existing problem.
14:01:44	adrianc	let me know if doing ifconfig <netdev> down on the source vf netdev solves the issue for you. (after migration)
14:01:49	sean-k-mooney	that is true but you could make it the base patch of the series so its easy to test all of them
14:02:13	sean-k-mooney	sure i need to restack but ill let you know in an hour or so
14:02:27	adrianc	sean-k-mooney: you will still need some POC code in neutron for multiple port binding
14:03:42	sean-k-mooney	ya i am useing that patch too
14:04:02	sean-k-mooney	devstack can deploy with unmerged patches from gerrit
14:04:23	sean-k-mooney	but it cant combine two sets fo unrelated patches
14:04:54	sean-k-mooney	so to test at the moment i jsut need to point to the top patch on the chain in nova and the neutron patch
14:05:26	adrianc	can be applied manually after deploy :)
14:06:12	sean-k-mooney	yep ill just create a branch locally and cherrypick them
14:06:18	sean-k-mooney	its fine
14:28:47	mriedem	melwitt: a change of mine hit the same vif plug timeout thing on that test with 7 ports http://logs.openstack.org/46/623246/3/check/nova-multiattach/efa830b/logs/screen-n-cpu.txt.gz?level=TRACE#_Dec_12_00_01_13_179474
14:29:36	mriedem	bauzas: can you look at this? https://review.openstack.org/#/c/615347/ - it drops old confusing compat which is actually contributing to lots of misleading warnings in the n-api logs and gate race failures
14:29:48	bauzas	mriedem: sure
14:29:52	sean-k-mooney	mriedem ya i have seen that a few times
14:33:43	mriedem	it's always the first port that we're missing
14:33:54	mriedem	we get 6 of the 7 network-vif-plugged events
14:33:59	sean-k-mooney	mriedem: one edgecase that i am awre of is in some configuration we start waithing for the network events too late
14:34:26	sean-k-mooney	we start waiting for the event just before we call plug on the interface
14:34:44	sean-k-mooney	for some neutron backend they send that event when we call bind
14:34:58	sean-k-mooney	i assume this is ml2/ovs however correct?
14:35:15	sean-k-mooney	that should not hit the race but i wonde if ther i a missed event in the logs
14:35:24	mriedem	yes ovs
14:37:32	sean-k-mooney	mriedem: http://logs.openstack.org/46/623246/3/check/nova-multiattach/efa830b/logs/screen-n-cpu.txt.gz?#_Dec_12_00_01_07_481084
14:37:39	mriedem	we plug it here
14:37:39	mriedem	Dec 11 23:56:11.855811 ubuntu-xenial-inap-mtl01-0001136812 nova-compute[29399]: INFO os_vif [None req-f43f3c35-dc0b-4aa4-bfd0-4046045b315e tempest-TaggedBootDevicesTest-250467933 tempest-TaggedBootDevicesTest-250467933] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:dd:9f,bridge_name='br-int',has_traffic_filtering=True,id=472ab433-bc18-41b5-85ae-009611117b70,network=Network(4988ef3d-6a8f-477f-b0a2-29
14:37:39	mriedem	019494),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap472ab433-bc')
14:38:20	mriedem	hmm wtf
14:38:39	mriedem	i also see this right before that port is plugged
14:38:39	mriedem	Dec 11 23:55:46.981194 ubuntu-xenial-inap-mtl01-0001136812 neutron-server[20899]: DEBUG neutron.notifiers.nova [-] Sending events: [{'tag': u'472ab433-bc18-41b5-85ae-009611117b70', 'name': 'network-changed', 'server_uuid': u'18e5987e-a535-4044-84aa-dd8f023cedcc'}] {{(pid=21001) send_events /opt/stack/new/neutron/neutron/notifiers/nova.py:245}}
14:39:31	mriedem	sean-k-mooney: that's a different port
14:39:35	mriedem	the unplugged event
14:39:55	sean-k-mooney	mriedem: yes i just noticed that
14:42:23	mriedem	asking in -neutron
14:53:58	frickler	nova stable cores: https://review.openstack.org/619254 is waiting for a second +2, if it gets that, I can start nagging mriedem about stable releases instead ;-)

Earlier   Later