Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-05-04
16:32:39 openstackgerrit melanie witt proposed openstack/nova master: Use run_immediately=True for _cleanup_running_deleted_instances https://review.opendev.org/657132
16:32:57 rm_work it sticks in "deleting" vm state
16:33:55 efried rm_work: But if the point is that this whole thing is centered around you wanting to delete the VM, and you're going to delete it as soon as that's possible, then you're fine.
16:34:13 efried The proc/mem/disk resources are just as leaked as the bandwidth resources in this case where the VM is undeletable.
16:34:25 rm_work though what happens if we *do* need to delete the port?
16:34:35 rm_work err sorry, if we do need to re-use the port?
16:34:44 rm_work it's the same bug, just unrelated to the placement-leakage issue
16:34:57 gibi_cape rm_work: how do you re-use the port when the compute is down?
16:34:59 rm_work (since you were asking about the specific nova bug)
16:35:07 rm_work move it to another compute
16:35:25 rm_work so the case would be if we are using one port as our main ingress (instead of AAP)
16:35:42 rm_work so if a compute host is powered off, we need to move that port over to another working VM
16:35:53 efried gibi_cape: Do we reclaim the allocations when a port is detached?
16:35:55 gibi_cape rm_work: if nova is not involved in this move then nova cannot make sure that the resource needs of the port also moved
16:35:56 rm_work but a detach will just fail to complete
16:36:14 gibi_cape efried: if the port is detached in nova then yes we delete the port allocation in placement
16:36:18 rm_work so we can't reattach the port to a different VM
16:36:26 efried rm_work: Right, it sounds like this scenario is not possible.
16:36:33 efried You can't detach it if you can't talk to the compute
16:36:38 efried did I understand correcctly?
16:36:46 rm_work ok, so that's not a solvable issue in nova?
16:37:24 rm_work what if that compute host never comes back up? in case of hardware failure, this is possible
16:37:28 rm_work is that port gone forever?
16:38:15 gibi_cape rm_work: if the compute never comes up then the resoruce leak in placement is not relevant as nobody else could you the leaked resource
16:38:17 mriedem zzzeek: sort of a random mysql uniqueness / primary key question for you when you have a sec
16:38:20 cfriesen new etherpad: https://etherpad.openstack.org/p/nova-ptg-train-2
16:38:27 rm_work right, not really talking about the placement-specific issue anymore
16:38:34 rm_work this is about the bug that was behind our reasoning
16:38:44 rm_work there's multiple impacts on our side from the same bug
16:39:02 rm_work one of them was that we just force-delete the port to handle cleanup on our side
16:39:12 rm_work but the other is that we necessarily lose the port
16:41:39 gibi_cape rm_work: I'm not fully understand what you really want to achieve with moving the bound port from one VM / compute to another. For me a neutron port feels like just a logical entity that easy to create / re-create
16:41:59 melwitt gibi_cape: they can't ever change IPs :)
16:42:06 gibi_cape melwitt: thanks
16:42:13 rm_work correct
16:42:25 rm_work we NEED to maintain the IP or we're sunk
16:44:33 kashyap stephenfin: I saw your comment scroll-by, on removing the needless libvirt version constants: I have a half-baked branch with some WIP. Or have you already posted something?
16:44:47 kashyap stephenfin: Trying to ensure we're doing duplicating work :-)
16:45:07 stephenfin kashyap: Nah, I'm not writing anything. Just reviewing your and mriedem's patches
16:45:16 kashyap stephenfin: Ah, like that
16:45:39 gibi_cape rm_work: if the VM on the down compute recovers later how that VM will be recovered from networking perspective? On the compute side the vif is still attached, but the neutron port is moved and attached to another VM.
16:45:41 kashyap stephenfin: All clear. Thanks, more clean-ups coming to a Firefox tab near you
16:45:53 rm_work gibi_cape: we don't care, that VM is going to be deleted anyway
16:46:01 rm_work once we move the port, we consider the old VM "dead" to us
16:46:18 rm_work we'll always be issuing a VM delete on it
16:46:34 rm_work very much cattle
16:47:11 gibi_cape rm_work: OK. then the policy thing could work. Octavia can do the port delete in neutron with the extra policy as Octavia makes sure the VM is deleted so the leaked resource is recovered by the VM delete
16:47:24 rm_work I think that would be the best approach
16:47:39 rm_work though it WOULD be awesome if we could figure out a solution to the underlying nova "bug"
16:47:55 rm_work I know you folks may not consider it a bug, but from our perspective, it sure seems that way
16:48:17 rm_work if a detach would just ... "complete" even if the compute host is unreachable, it'd make us very happy :)
16:48:29 gibi_cape rm_work: would be nice to have a bug report on that or / and talk about this in the nova room
16:49:09 rm_work ok -- though i believe we've brought it up before and were told that was expected behaviour
16:49:28 rm_work but I suppose filing it officially would at least provide history for a decision even if it's still the same
16:50:16 gibi_cape rm_work: I don't have the historical context
16:50:31 gibi_cape rm_work: so I agree that written bug report would be nice
16:51:25 openstackgerrit Chris Dent proposed openstack/nova master: Make nova-tox-functional-py36 reusable https://review.opendev.org/657074
16:51:51 rm_work we'll make sure to write something up, hopefully today
16:52:27 gibi_cape rm_work: thanks
16:55:23 mriedem sean-k-mooney: mnaser: i might have figured out a reason for the port binding update failure that rcolvin was hitting https://bugs.launchpad.net/nova/+bug/1822884/comments/21
16:55:25 openstack Launchpad bug 1822884 in OpenStack Compute (nova) "live migration fails due to port binding duplicate key entry in post_live_migrate" [Undecided,In progress] - Assigned to sean mooney (sean-k-mooney)
16:55:42 mriedem i have a recreate for it in my nova-multi-cell resize change
16:57:49 gibi_cape rm_work: replied on ML too
17:03:03 rm_work yeah, that seems right to me
17:07:56 shuquan rbd disks: convert from source format to raw https://review.opendev.org/#/c/640271/ https://review.openstack.org/#/c/642667/
17:10:53 openstackgerrit Balazs Gibizer proposed openstack/nova master: Log when port resource is leaked during port delete https://review.opendev.org/657079
17:25:57 openstackgerrit Tushar Patil proposed openstack/nova-specs master: Support filtering of allocation_candidates by forbidden aggregates https://review.opendev.org/609960
17:27:56 kashyap efried: cfriesen; johnthetubaguy: Thanks, folks!
17:28:45 cfriesen can anyone point me at what's breaking the nova-live-migration test for https://review.opendev.org/#/c/621646/ ? I can't tell if it's an issue with my patch or a zuul issue.
17:29:41 mriedem cfriesen: http://logs.openstack.org/46/621646/21/check/nova-live-migration/8a081f1/logs/devstacklog.txt.gz#_2019-05-02_22_32_53_983
17:29:49 mriedem http://status.openstack.org/elastic-recheck/#1827083
17:30:24 cfriesen mriedem: thanks
17:43:18 openstackgerrit Merged openstack/nova-specs master: Support filtering of allocation_candidates by forbidden aggregates https://review.opendev.org/609960
17:47:55 openstackgerrit Balazs Gibizer proposed openstack/nova master: pull out functions from _heal_allocations_for_instance https://review.opendev.org/655457
17:47:56 openstackgerrit Balazs Gibizer proposed openstack/nova master: Prepare _heal_allocations_for_instance for nested allocations https://review.opendev.org/637954
17:47:56 openstackgerrit Balazs Gibizer proposed openstack/nova master: reorder conditions in _heal_allocations_for_instance https://review.opendev.org/655458
17:47:57 openstackgerrit Balazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations https://review.opendev.org/637955
17:47:57 openstackgerrit Balazs Gibizer proposed openstack/nova master: pull out put_allocation call from _heal_* https://review.opendev.org/655459
17:49:19 openstackgerrit Merged openstack/nova master: Improve test coverage of nova.privsep.utils. https://review.opendev.org/655281
17:49:21 kashyap cfriesen: Do you know what johnthetubaguy meant by the phrase "fragment hypervisor"?
17:49:46 openstackgerrit Dan Smith proposed openstack/nova master: Enable image type query support in nova-next https://review.opendev.org/656903
17:49:46 openstackgerrit Dan Smith proposed openstack/nova master: Add image type request filter https://review.opendev.org/656413
17:49:47 openstackgerrit Dan Smith proposed openstack/nova master: Add docs for image type support request filter https://review.opendev.org/657025
17:51:39 cfriesen kashyap: nope. :)
17:54:32 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Rework the PCI passthrough guides https://review.opendev.org/635243
18:31:57 sean-k-mooney dansmith: https://etherpad.openstack.org/p/nova-ptg-train-governance
20:08:14 openstackgerrit Merged openstack/nova stable/stein: libvirt: Always disconnect volumes after libvirtError exceptions https://review.opendev.org/657109
20:32:17 openstackgerrit Merged openstack/nova master: Add image type capability flags and trait conversions https://review.opendev.org/652710
21:08:46 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.opendev.org/575311
21:08:55 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9) https://review.opendev.org/575581
21:09:02 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10) https://review.opendev.org/576017
21:09:15 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11) https://review.opendev.org/576018
21:09:32 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12) https://review.opendev.org/576019
21:09:46 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020
21:10:01 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027
21:10:17 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031
21:10:33 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299
21:10:51 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344
21:11:06 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673
21:11:17 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676

Earlier   Later