Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-08-25
14:15:48 rs_goyal__ @gibi
14:15:53 sean-k-mooney you were fixing issue we had with persitent xml and transient getting out of sync but maybe the locking will also help
14:16:29 sean-k-mooney aarents: there is potentially a seperate cause to that bug
14:16:37 sean-k-mooney https://bugs.launchpad.net/nova/+bug/1892870
14:16:37 openstack Launchpad bug 1892870 in OpenStack Compute (nova) "concurrent interface detach/attach may accidently unbind port" [Undecided,In progress] - Assigned to Alexandre arents (aarents)
14:17:32 sean-k-mooney aarents: Detaching interface %(mac)s failed because the device is no longer found
14:17:34 sean-k-mooney on the guest.
14:18:00 sean-k-mooney this can happen for other reasons then a race at lest that seamed to be the case for volumes
14:19:03 sean-k-mooney aarents: that siad i dont nessacarly thing adding a lock is a bad thing although your commit mentions
14:19:05 sean-k-mooney multiple detach/attach are run
14:19:08 sean-k-mooney concurrently
14:19:12 sean-k-mooney but you are only locking detach
14:19:31 sean-k-mooney you might want to use a shared lock between attach and detach
14:19:44 sean-k-mooney and add a lock to attach too
14:20:00 lyarwood sean-k-mooney: yeah I think we already lock for disks
14:20:24 aarents sean-k-mooney: Yes because the race exists only because of detach that sends multiple UNBIND
14:20:49 aarents if it is fixed on detach attach should be consistent I think
14:20:51 sean-k-mooney what happens if we do multiple attach api calls concurrently
14:21:39 sean-k-mooney aarents: since the syconisation woudl be on the interface uuid i think it would be fairly cheap to just lock both operation
14:23:23 aarents sean-k-mooney: hum why not
14:23:42 sean-k-mooney lyarwood: and yes we already lock in detach_volume using the volume_uuid
14:24:22 aarents lock only on interface uuid not instance_uuid-interface_uuid but on bot attach and detach ?
14:24:59 sean-k-mooney so volumes lock on both attach and detach
14:25:47 sean-k-mooney i would change teh lock name from 'detach_interface-%s-%s' to 'interface-%s-%s'
14:26:16 sean-k-mooney you can pass the instance uuid and port but honestly the neutron port uuid will be unique so the instance uuid is not actully needed
14:26:53 aarents sean-k-mooney and put it on both detach and attach ?
14:27:02 sean-k-mooney yep
14:27:06 aarents make sense
14:27:11 sean-k-mooney to keep it consitent with volumes
14:27:20 aarents yep
14:27:39 sean-k-mooney volumes actully does @utils.synchronized(instance.uuid)
14:27:50 sean-k-mooney but i prefer using the port uuid
14:28:14 sean-k-mooney unless we want to block detaching differnet ports in parallel
14:28:40 sean-k-mooney i guess that is why the instance uuid is used for volumes to prevent detaching multiple volumes in paralel
14:28:45 sean-k-mooney but im not sure that is actully required
14:29:22 aarents sean-k-mooney: yeah but I want to keep it // I make some test
14:30:00 sean-k-mooney ya using a uniqe lock name for this is fine
14:30:43 sean-k-mooney if at some point we really need to make it instance wide we can just drop the port uuid
14:30:50 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Provide default 'host_info' for '_get_connection' https://review.opendev.org/746942
14:30:50 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add 'start_compute' helper https://review.opendev.org/746943
14:30:51 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Use tempdir for CONF.instances_path https://review.opendev.org/746944
14:30:51 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Clean up PCI tests https://review.opendev.org/746945
14:30:52 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add resize tests for PCI servers https://review.opendev.org/746946
14:30:52 openstackgerrit Stephen Finucane proposed openstack/nova master: conductor: Don't use setattr https://review.opendev.org/746947
14:30:53 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add test for SR-IOV neutron ports https://review.opendev.org/746948
14:30:53 openstackgerrit Stephen Finucane proposed openstack/nova master: fixtures: Handle binding of first port https://review.opendev.org/746949
14:30:54 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add live migration tests for PCI, SR-IOV servers https://review.opendev.org/746950
14:31:19 aarents sean-k-mooney, I made some test reauest 7 detach on same instance taking 4s to detach with a lock per instance-port and > 10sec when only instance
14:32:27 sean-k-mooney yep as long as libvirt safely handels the parrale detach request we dont need to lock at the instance levle
14:32:35 gibi sean-k-mooney, aarents: good catch about the parallel detach, I queued up the fix in my review list
14:32:53 aarents So I prefer 'interface-%s-%s' on both
14:32:55 sean-k-mooney gibi: aarents found it :)
14:33:03 aarents gibi: thanks
14:33:10 gibi then aarents gets the prize :)
14:42:51 aarents That one of my two downstream summer bugs :P other one will come soon..
14:43:58 gibi I had summer vacation, you had summer bugs. I'm feeling lucky :D
14:44:19 aarents :D
14:52:10 rs_goyal__ @gibi @artom : I tried tagging the network, but the interface is getting created like I explained above. I used `nova interface-attach --net-id <network> --tag priv <instance>`for attaching a newtork interface on an existing vm.
14:53:34 artom rs_goyal__, the actual interface names (eth0, ens1, etc) aren't affected
14:53:42 artom rs_goyal__, you need to look in the metadata
15:00:28 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747969
15:01:16 rs_goyal__ @artom : okay. thanks.
15:03:05 gibi lyarwood: +2
15:03:29 openstackgerrit Lee Yarwood proposed openstack/nova stable/ussuri: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747972
15:03:29 lyarwood gibi: thanks
15:03:36 lyarwood stephenfin: ^ master change is up btw
15:03:47 gibi lyarwood: I thank you to taking care of this
15:04:22 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747973
15:04:45 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747974
15:04:46 lyarwood gibi: np :)
15:06:29 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747975
15:07:47 stephenfin lyarwood: done \o/
15:07:52 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747976
15:08:28 openstackgerrit Stephen Finucane proposed openstack/nova master: Add generic reproducer for bug #1879878 https://review.opendev.org/747744
15:08:28 openstack bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,In progress] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane)
15:08:28 openstackgerrit Stephen Finucane proposed openstack/nova master: Don't unset Instance.old_flavor, new_flavor until necessary https://review.opendev.org/744958
15:08:28 openstackgerrit Stephen Finucane proposed openstack/nova master: Move confirm resize under semaphore https://review.opendev.org/747745
15:08:28 openstackgerrit Stephen Finucane proposed openstack/nova master: Move revert resize under semaphore https://review.opendev.org/747746
15:08:29 openstackgerrit Stephen Finucane proposed openstack/nova master: Cleanup 'drop_move_claim' and '_drop_move_claim' https://review.opendev.org/747747
15:08:29 openstackgerrit Stephen Finucane proposed openstack/nova master: manager: Address TODO https://review.opendev.org/747748
15:08:30 openstackgerrit Stephen Finucane proposed openstack/nova master: manager: Move context manager up one level https://review.opendev.org/745674
15:08:30 openstackgerrit Stephen Finucane proposed openstack/nova master: compute: Add type hints for resize functions https://review.opendev.org/745341
15:08:31 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: compute: Add more type hints for resize functions https://review.opendev.org/745675
15:09:19 stephenfin artom: that failure in https://review.opendev.org/747744 is weird. Looks like the mock was being applied globally or something :-\
15:09:29 stephenfin so I just removed the mock
15:09:39 artom stephenfin, yeah, I can't reproduce it locally when running just that one single test
15:09:50 artom But maybe it's somehow related
15:10:20 openstackgerrit Lee Yarwood proposed openstack/nova stable/pike: libvirt: Provide VIR_MIGRATE_PARAM_PERSIST_XML during live migration https://review.opendev.org/747978
15:21:02 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Enable real policy fixture by default https://review.opendev.org/745906
15:21:02 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add and use 'GlanceFixture' https://review.opendev.org/745907
15:21:03 openstackgerrit Stephen Finucane proposed openstack/nova master: tests: Remove '_FakeImageService' https://review.opendev.org/745908
15:25:58 frickler artom: rs_goyal__: there's some work in progress on adding tag support to OSC, help appreciated. https://review.opendev.org/#/q/topic:story/2002195+(status:open+OR+status:merged)
15:28:43 frickler but I also admit that I didn't know about this feature up to now, which might help with some of my usecases, too, guess I need to take a deeper look anyhow. guess nobody did build cloud-init support for it, either
15:36:25 artom frickler, so I keep hearing that osc is moving to using the sdk... is that even true?
15:36:54 artom Because I started an effort to bring sdk up to speed (from the nova pov at any rate), the logic being that's the more "future-proof" place to invest work
15:37:11 artom But then I keep seeing stuff being added to osc directly...
15:37:31 artom Is there even a grand plan, or is this all ad-hoc patches?
15:39:06 openstackgerrit Merged openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029
15:43:17 stephenfin lyarwood: reviewed the nova-image-download-via-rbd bp
15:44:05 stephenfin artom: not frickler, wdym by "stuff [being added] to osc directly"

Earlier   Later