Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-25
05:35:29 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.openstack.org/576020
05:35:37 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.openstack.org/576020
05:35:56 openstackgerrit Tony Breeds proposed openstack/nova stable/queens: Don't persist zero allocation ratios in ResourceTracker https://review.openstack.org/613271
05:36:10 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.openstack.org/576027
05:36:12 openstackgerrit Tony Breeds proposed openstack/nova stable/queens: Update resources once in update_available_resource https://review.openstack.org/612294
05:36:18 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.openstack.org/576027
05:37:37 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.openstack.org/576031
05:37:45 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.openstack.org/576031
05:38:36 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.openstack.org/576299
05:38:44 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.openstack.org/576299
05:39:43 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.openstack.org/576344
05:39:54 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.openstack.org/576344
05:40:29 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.openstack.org/576673
05:40:36 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.openstack.org/576673
05:41:14 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.openstack.org/576676
05:41:25 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.openstack.org/576676
05:42:11 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.openstack.org/576689
05:42:20 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.openstack.org/576689
05:42:39 openstackgerrit Tony Breeds proposed openstack/nova stable/pike: Document unset/reset wrinkle for *_allocation_ratio options https://review.openstack.org/647292
05:43:03 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.openstack.org/576709
05:43:13 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.openstack.org/576709
05:43:33 openstackgerrit Tony Breeds proposed openstack/nova stable/pike: Don't persist zero allocation ratios in ResourceTracker https://review.openstack.org/613263
05:43:51 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.openstack.org/576712
05:43:54 openstackgerrit Tony Breeds proposed openstack/nova stable/pike: Update resources once in update_available_resource https://review.openstack.org/612295
05:44:03 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.openstack.org/576712
06:19:58 openstackgerrit Abhishek Kekane proposed openstack/nova-specs master: Support multiple backend of Glance https://review.openstack.org/641210
06:58:11 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add a live migration regression test https://review.openstack.org/641200
06:59:58 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.openstack.org/576758
07:00:04 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.openstack.org/576758
08:22:35 openstackgerrit Martin Schuppert proposed openstack/nova stable/stein: Eventlet monkey patching should be as early as possible https://review.openstack.org/647310
09:40:01 openstackgerrit Surya Seetharaman proposed openstack/nova-specs master: Support adding the reason behind a server lock https://review.openstack.org/638629
10:57:44 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add falvor to requested_resources in RequestSpec https://review.openstack.org/647396
11:34:58 openstackgerrit Merged openstack/nova-specs master: Replace openstack.org git:// URLs with https:// https://review.openstack.org/646695
12:36:46 shilpasd Hi: openstacksdk returning 400 for 'force_service_down' API with version as 2.72, pl refer https://bugs.launchpad.net/ubuntu/+source/python-openstacksdk/+bug/1821584
12:36:47 openstack Launchpad bug 1821584 in python-openstacksdk (Ubuntu) "Compute 'force_service_down' API returns 400 with current version 2.72 " [Undecided,New]
12:36:49 tssurya melwitt, mriedem: I feel we should fallback to legacy if the queued_for_delete is not migrated and is still NULL like its done for user_id : https://review.openstack.org/#/c/638072/14/nova/objects/instance_mapping.py@386
12:37:25 shilpasd Hi All: please confirm the same
12:58:38 efried shilpasd: Thanks for the bug report.
12:59:00 efried FYI there's a community-wide goal getting set up for Train to try to get more of the CLIs up to parity
12:59:27 efried If you have the means, it would be great if you wanted to propose a fix for this.
13:00:07 shilpasd <efried> sure
13:00:16 shilpasd efried: sure
13:00:18 sean-k-mooney efried: in this case its an openstacsdk limitation not a cli issue
13:00:33 efried sean-k-mooney: Yes, I agree. shilpasd opened the bug against the right component.
13:00:41 sean-k-mooney nova cli and osc do not use the sdk for the api currently
13:00:56 sean-k-mooney ah yes they did
13:02:45 sean-k-mooney efried: stephenfin and i were talking internally about ho to improve cli partity
13:04:00 sean-k-mooney efried: we were thinking a phased approch make the most sence. first make osc have partity with nova cli by just delegating to it internall. then port osc to openstack sdk once that is done
13:04:28 sean-k-mooney so form a user point of view we get to cli parity first then we can get teh sdk to partity and swap over to useing it
13:05:10 sean-k-mooney but there are still big gaps in that plan. e.g. should clint only things like nova host-evac be ported and how to deal with micorverions
13:07:57 ganso the Placement API via curl. After several migration attempts that fail in the same spot, the source node is filled with these allocations that prevent new instances from being created or instances migrated to this node.
13:07:57 ganso Hi folks. I am about to log a bug and would like to confirm if this is an intentional behavior or not: If an exception is raised by the driver during confirm_migration, the migration record is stuck in confirming state and the allocations are not removed. The instance is fine at the destination in this stage, but the source host has allocations that is not possible to clear without going to the database or invoking
13:08:39 sean-k-mooney efried: but ya in anycase i do hope we can make progress on that goal in train
13:16:18 efried ganso: What release are you running?
13:16:41 ganso efried: I tried pike, queens and stein
13:17:07 ganso efried: same problem in all of them ^
13:18:50 efried ganso: Well, that doesn't sound great. mriedem is probably your best bet to say whether this is a known issue. If he doesn't chime in here I think you can safely open a bug.
13:19:34 ganso efried: thanks! I will wait for mriedem's confirmation first.
13:20:58 mriedem looking, but i wouldn't be at all surprised
13:21:58 mriedem ack on the migration.status not being set to error because the @errors_out_migration_ctx decorator is not used here https://github.com/openstack/nova/blob/926753ee6bebee5bacc29073e123ed5c1a0bb647/nova/compute/manager.py#L3933
13:22:11 mriedem so you never get this far https://github.com/openstack/nova/blob/926753ee6bebee5bacc29073e123ed5c1a0bb647/nova/compute/manager.py#L4011
13:22:22 mriedem nor do the allocations get cleaned up https://github.com/openstack/nova/blob/926753ee6bebee5bacc29073e123ed5c1a0bb647/nova/compute/manager.py#L4017
13:22:32 mriedem so yes it's a big
13:22:38 mriedem setting the migration status to error is easy
13:22:48 mriedem knowing whether or not the allocations should be changed is harder
13:23:22 mriedem ganso: the instance status goes to ERROR right? https://github.com/openstack/nova/blob/926753ee6bebee5bacc29073e123ed5c1a0bb647/nova/compute/manager.py#L3993
13:23:32 ganso mriedem: yes the instance goes to error
13:23:33 mriedem which means you can't revert the resize after confirm fails
13:23:45 mriedem unless you manually reset the vm_state to 'resized' in the db
13:24:16 mriedem so at this point, the instance is on the dest, the db points the instance at the dest, and confirm fails on the dest and you're stuck in error state,
13:24:17 ganso mriedem: I didn't try that. I noticed I could only change the status to either error or active, so I changed the status to active and could use the instance fine after that, even migrate it again.
13:24:25 mriedem i would say we probably just consider the instance is stuck there now and cleanup the allocations
13:24:39 mriedem ganso: yeah reset status only changes to active/errror
13:24:40 mriedem *error
13:24:53 mriedem so you could have tried hard rebooting the instance to fix it also since you can do that with an error instance
13:25:06 mriedem but that's not going to cleanup the allocations as you found
13:25:17 mriedem ganso: anyway, it's definitely a bug
13:25:24 ganso mriedem: hmmm seems to me the migration_confirm runs on the source, and fails there. The libvirt driver only cleans up on that method's implementation
13:25:51 mriedem oh right, yeah confirm_resize runs on the source host
13:25:54 mriedem to cleanup the source host
13:26:07 efried no confirm_resize --force?
13:26:18 efried "I know this thing is copacetic on the destination, make it so"
13:26:29 mriedem probably another good reason to just cleanup the guest on the hypervisor as best effort but still cleanup the allocations
13:26:43 ganso mriedem: thanks! I will log the bug =)
13:26:47 mriedem efried: that would assume you know you need to force it before the instance gets stuck in error status
13:27:01 mriedem you can't confirm resize on an error instance
13:27:10 efried That's what the force would be
13:27:20 mriedem ew
13:27:21 efried "I done checked the instance, and it's good. Make it look good in the databases and stuff"
13:27:30 efried shrug, just spitballing.
13:27:38 mriedem you can fix it up with a hard reboot
13:27:41 mriedem on the dest
13:27:44 efried except the allocations
13:27:48 mriedem but that won't clean the allocations
13:28:05 mriedem right, so what i'm saying is we should suck less and care for where errors can happen in a more granular way
13:28:12 ganso efried, mriedem: yea the main problem are the allocations.
13:28:27 efried does nova manage heal_allocations work on something like this?
13:28:28 mriedem these giant compute / conductor manager methods are so convoluted that it gets hard to deal with errors in isolation
13:28:31 mriedem and if they should be fatal or not
13:28:53 mriedem efried: no, that only creates missing allocations, doesn't cleanup ones that shouldn't exist

Earlier   Later