Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-05-25
10:39:39 kashyap sean-k-mooney: For test configuration, right: wherever 'ide' is used, that needs to be replaced (or better: conditionalized). E.g. https://review.opendev.org/#/c/721555/
10:39:53 sean-k-mooney ERROR cinder.volume.api [None req-a5e3e7e0-f8ce-4a58-bd74-53517edc5fd5 tempest-VolumeQuotasNegativeTestJSON-777903610 tempest-VolumeQuotasNegativeTestJSON-777903610] Quota exceeded for a769409cccff4ea48d0fe2efea551c8a, tried to extend volume by 1G, (1G of 1G already consumed).: cinder.exception.OverQuota: Quota exceeded for resources: ['gigabytes']
10:40:01 sean-k-mooney i think the issue are mostly down to that
10:40:41 sean-k-mooney kashyap: i dont think ide will be used anyhwere in the test and that is not the reason for the failures as far as i can tell
10:41:16 kashyap sean-k-mooney: IDE _is_ used in some places, see the link I posted :-)
10:41:32 kashyap [But be back in a few...]
10:41:37 sean-k-mooney well most of the test failures are down to exceeding quota i belive
10:42:15 sean-k-mooney why did lee us ide for hw_rescue_bus?
10:42:43 sean-k-mooney that specificly was only added last cycle so that should have used ide form the start
10:48:42 stephenfin yo, bauzas - care to look at https://review.opendev.org/#/c/728968/ and surrounding patches?
11:04:25 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: remove get_quota_classes() driver method https://review.opendev.org/615615
11:04:26 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: remove Context.quota_class https://review.opendev.org/615616
11:04:26 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: remove FakeContext from quota unit tests https://review.opendev.org/615617
11:04:27 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: remove _no_class tests https://review.opendev.org/615618
11:04:27 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: clean up DbQuotaDriver unit tests https://review.opendev.org/615619
11:04:28 openstackgerrit Stephen Finucane proposed openstack/nova master: quota: rename arguments to clarify they are limits https://review.opendev.org/615633
11:27:23 openstackgerrit Balazs Gibizer proposed openstack/nova stable/train: Add admin doc information about image cache resource accounting https://review.opendev.org/730326
11:54:06 sean-k-mooney stephenfin: i forget tox -e py3 worked
12:03:36 gibi sean-k-mooney: I know that you have a better view than me regarding network outage during live migration. there is a new bug 1880389 reported about it. Could you check it if you have time?
12:03:36 openstack bug 1880389 in OpenStack Compute (nova) "lost net connection when live migration" [Undecided,New] https://launchpad.net/bugs/1880389
12:09:41 sean-k-mooney sure i can take a look shortly just grabing coffee but i have it open for when i get back
12:10:23 sean-k-mooney also without looking is sure bad things proably happen if there is a network outage while live migrating
12:16:18 sean-k-mooney ok so the issue is with the vm losing network connectivyt not a network partion at the host level causeing the migration to fail in weird ways which is less scary
12:18:19 sean-k-mooney gibi: i need to dig deaper but its proably a duplicate of https://bugs.launchpad.net/neutron/+bug/1815989 which is cause by the root cause of https://bugs.launchpad.net/neutron/+bug/1734320
12:18:19 openstack Launchpad bug 1815989 in OpenStack Compute (nova) "OVS drops RARP packets by QEMU upon live-migration causes up to 40s ping pause in Rocky" [Medium,In progress]
12:18:20 openstack Launchpad bug 1734320 in OpenStack Compute (nova) "Eavesdropping private traffic" [Undecided,In progress] - Assigned to sean mooney (sean-k-mooney)
12:19:05 bauzas stephenfin: ack will look
12:19:19 sean-k-mooney gibi: https://review.opendev.org/#/c/724386/ is a potential workaround which some have said fixes the issue
12:19:48 sean-k-mooney gibi: this is how i actully wanted to fix it https://review.opendev.org/#/q/status:open++branch:master+topic:bug/1734320
12:20:14 sean-k-mooney that said they are using linux bridge so it could be something else
12:22:00 sean-k-mooney "the ping was broken down. and the ping package from the physic switch was still set to the source."
12:22:28 sean-k-mooney so they were observing that pings are switched to the souce host which implies the mac learning frames were lost
12:22:55 sean-k-mooney which is bug: #1815989
12:22:55 openstack bug 1815989 in OpenStack Compute (nova) "OVS drops RARP packets by QEMU upon live-migration causes up to 40s ping pause in Rocky" [Medium,In progress] https://launchpad.net/bugs/1815989
12:23:14 sean-k-mooney although in this case they are reportign the same behavior for linux bridge
12:23:53 sean-k-mooney ah last paragraph "And I found the time of 5 RARP from VM is before the time of the vlan sub interface insert to the linuxbridge bridge when live migration. maybe this will help us to make sure the problem I found."
12:24:19 sean-k-mooney so yes its the same underlying issue
12:25:19 sean-k-mooney the issue happens in the ovs case when libvirt recreated the port that os-vif has allready created resulting in a race with the neutron l2 agent
12:25:38 sean-k-mooney i did not think we had the same race for linux bridge but ill take a look
12:26:04 gibi sean-k-mooney: thanks
12:48:15 sean-k-mooney so we use interface type bridge for linux bridge which is not very surpising
12:48:18 sean-k-mooney <interface type='bridge'>
12:48:20 sean-k-mooney <source bridge='br1'/>
12:48:22 sean-k-mooney <target dev='vnet7'/>
12:48:24 sean-k-mooney <mac address="00:11:22:33:44:55"/>
12:48:26 sean-k-mooney </interface>
12:48:48 sean-k-mooney and se the target, mac and source to the values provided by neutron
12:49:41 sean-k-mooney before the vm is spawned on the destination host we invoke os-vif to ensure that the bridge exists
12:50:29 sean-k-mooney so when we define the libvirt domian there should be nothing for libvirt to do it should be alredy configured
12:51:06 sean-k-mooney this is all os-vif does https://github.com/openstack/os-vif/blob/master/vif_plug_linux_bridge/linux_bridge.py#L97-L112
12:52:45 openstackgerrit Merged openstack/os-vif master: Remove .testr.conf https://review.opendev.org/729156
12:55:53 sean-k-mooney i should proably go clean up that code since nova net is now dead and we should not really be doing any ip filtering in os-vif even in neutron is not anymore since that was really for nova net
13:00:01 gibi sean-k-mooney: thanks for triageing that bug
13:05:28 sean-k-mooney im debating if im goint to try and repoduce it or not
13:06:18 sean-k-mooney i would need to check the libvirt sorce code but i dont think it should have the same race
13:06:32 sean-k-mooney it might but i did not think it should
13:32:16 openstackgerrit Stephen Finucane proposed openstack/nova master: virt: Add 'context', drop 'network_info' parameters for 'unrescue' https://review.opendev.org/730382
13:32:17 openstackgerrit Stephen Finucane proposed openstack/nova master: Add emulated TPM support to Nova https://review.opendev.org/631363
13:32:17 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934
13:51:11 openstackgerrit sean mooney proposed openstack/os-vif master: update tox envs and support pdf docs https://review.opendev.org/728037
14:02:11 sean-k-mooney stephenfin: hopefully you are more happy with ^
14:02:49 sean-k-mooney but also that is more a merge when convenient patch. as long as we merge it before m3 (perferably m1) its fine
14:02:53 sean-k-mooney so no rush
14:03:22 stephenfin better but still not correct :)
14:04:26 sean-k-mooney hehe ok im goint to update my osc patch for evacuate ill take a look when that is done
14:09:02 sean-k-mooney just looked. ya ill do some of the chagne in a follow up and adress some in that patch. ill push both patches togeter this evening or tommorow
14:09:25 sean-k-mooney that patch is getting a bit long but the change you suggest make sense
14:18:52 sean-k-mooney 'Wait for instance evacuation to complete' sound less worng then 'Wait for evacuation to complete' right?
14:20:01 stephenfin The operation affects multiple instances, right?
14:20:10 sean-k-mooney no
14:20:13 sean-k-mooney only one instance
14:20:21 stephenfin then the first one is better, yes
14:20:28 sean-k-mooney im porting evacuate not host-evacuate
14:20:48 stephenfin although I don't know if osc uses the term "instance"
14:20:59 sean-k-mooney host-evacuate it the clinet side command that enumarate the instance on the host and evacuates all of them
14:21:20 sean-k-mooney i think it might use server but that is kind of wronge
14:21:23 sean-k-mooney i could use server
14:22:02 sean-k-mooney server is proably more inline with the other messages in teh file so ill use that
14:33:02 openstackgerrit Merged openstack/python-novaclient master: Remove Babel requirement https://review.opendev.org/723139
15:37:42 openstackgerrit Balazs Gibizer proposed openstack/nova-specs master: Make Feature Liaison optional https://review.opendev.org/730638
16:57:52 openstackgerrit Stephen Finucane proposed openstack/nova master: virt: Add 'context', drop 'network_info' parameters for 'unrescue' https://review.opendev.org/730382
16:57:52 openstackgerrit Stephen Finucane proposed openstack/nova master: Add emulated TPM support to Nova https://review.opendev.org/631363
16:57:53 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934
23:28:38 openstackgerrit Merged openstack/nova stable/queens: Revert "nova shared storage: rbd is always shared storage" https://review.opendev.org/726864
#openstack-nova - 2020-05-26
01:10:02 openstackgerrit Wenping Song proposed openstack/nova master: WIP: Cyborg suspend/resume support https://review.opendev.org/729945
02:47:15 openstackgerrit Merged openstack/nova master: docs: Resolve issue with deprecated extra specs https://review.opendev.org/726315
04:08:56 openstackgerrit Merged openstack/nova master: replace the "hide_hypervisor_id" to "hw:hide_hypervisor_id" https://review.opendev.org/722187
05:14:50 tacco ho everyone. Have a mostly clean setup. But now i have a problem with downsizing of a VM.
05:15:28 tacco i cannot apply a smaller flavor anymore, but there is still a bunch of space on HVs and it is also allowed to migrate to the same host.
05:15:34 tacco have no clue what is broken in here.
05:15:41 tacco Will the RAM allocated two times?
05:15:59 tacco because it looks as it would be the RAM that causes this issue.
05:16:04 tacco any ideas?
06:55:31 gibi good morning nova
07:09:38 bauzas good morning Nova
07:10:11 openstackgerrit Wenping Song proposed openstack/nova master: WIP: Cyborg suspend/resume support https://review.opendev.org/729945
07:20:59 slaweq gibi: bauzas: good morning nova (from neutron) ;)
07:24:48 gibi slaweq: :)
07:42:00 tacco morning everyone. :)
07:42:52 tacco anyone has a idea why placement shows targets for possible placement of a large flavor. but... downsize a vm to a smaler flavor is not possible? any hints? tips and tricks?

Earlier   Later