Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-03-25
15:41:22 brinzhang because the BASE_POLICY_NAME % 'events:details' just limit show details in 'events' dict
15:41:45 luyao lyarwood: we already have vpmem cleanup logic inside libvirt driver, driver.cleanup will invoke vpmem cleanup.
15:42:37 lyarwood luyao: so why can't we just call that specific logic from other places instead of calling the entire cleanup method?
15:43:21 luyao lyarwood: if I want to cleanup vpmems on destination host but do_cleanup is False, I need a rpc call for vpmem cleanup
15:43:47 gmann brinzhang: yeah i get that but my point is it is difficult for operator to decide that 'details' (which can have non-nova exception so does leak the infa info) can be shown to non-admin and traceback not.
15:44:46 luyao lyarwood: alternatively we can set do_cleanup to True, then rpc call rollback_live_migration_at_destination will be invoked, then vpmem cleanup will be called inside that
15:45:02 brinzhang gmann: if the details is non-nova exception, it will be just only show the Exception class name to the details
15:45:16 gmann brinzhang: if any operator ask how to use these two policy in different way (one allowed for admin and one for non-admin) then we would not have clear answer right ?
15:45:36 brinzhang gamnn: pls see https://review.opendev.org/#/c/712697/
15:48:13 luyao lyarwood: so I asked could I set do_cleanup to True if there are vpmems
15:48:18 brinzhang gmann: 'traceback' show the exception details info, contains python path, and the all details. but 'details' just only show the format message if it's a nova exception, but if that is an non-nova excetption, we just show the simple info the the non-admin
15:48:52 brinzhang I donnot think it no clear
15:49:04 luyao lyarwood: Do I make it clear?
15:49:14 lyarwood luyao: yeah okay, that might be okay in the short term but I think after this we really need to clean this interface up
15:49:36 gmann brinzhang: that is what i was thinking to add in API side but serialize_args does.
15:49:49 lyarwood luyao: cleanup within libvirt is actually looking at migrate_data so why we are making the call to cleanup dependent on it is weird
15:49:51 gmann to handle the non nova exception details
15:49:53 luyao lyarwood: yeah agree
15:52:23 brinzhang gmann: you mean, something need I add in os-instance-action API?
15:54:53 gmann brinzhang: no i mean hiding detail about non nova exception but exception name itself can leak few info about driver used etc.
15:55:17 gmann can non-admin take action based on non-nova exception ?
15:56:10 brinzhang maybe try to do something that they can do, nothing else
15:56:11 gmann i am thinking if we hide the non- nova exception from 'details' field and only expose the nova exception which is what use case of 'details' is for non-admin
15:56:39 luyao lyarwood: we only have instance path file to cleanup previously but now we have other devices needs cleanup
15:56:54 gmann dansmith: ^^ ? any use case of keeping non-nova exception name in action event 'details' field.
15:57:07 gmann admin anyways can see all details from traceback
15:58:23 brinzhang gmann: thanks, I am sorry it's too later for me, I have to go.
15:58:31 gmann so that we can keep new field 'details' usable and no info leak for non-admin
15:58:58 gmann brinzhang: ah sorry. yeah. I will reply on review. thanks for discussion and late night.
15:59:38 luyao lyarwood: we can also add a flag in libvirt migrate data to tell if there are vpmems needs cleanup, I'm not sure is it necessary?
15:59:45 brinzhang gmann:We only show non-nova exception class name to users, I don't think it will cause serious information leakage.
16:01:03 brinzhang gmann: this serialize_args change comes mriedem and dansmith, if they are around, I think you can get more.
16:01:09 brinzhang gmann: thanks too, bye
16:01:32 sean-k-mooney luyao: we had to do host cleanup before for things other then the instnace files
16:02:04 sean-k-mooney luyao: like removing mounted volumes, cleaning up ports or other actions
16:03:08 brinzhang gmann: this is the original thought https://review.opendev.org/#/c/694428/9/nova/objects/instance_action.py@196
16:03:33 openstackgerrit Lee Yarwood proposed openstack/python-novaclient master: Microversion 2.83 - Stable device boot from volume rescue https://review.opendev.org/714956
16:04:01 lyarwood luyao: possibily, just need to jump on a call and I'll try to update the review again
16:04:11 sean-k-mooney brinzhang: for non admin i think they should only see the class name for nova exceptions too
16:04:53 brinzhang sean-k-mooney: yeah, agree, make sense to me too.
16:05:35 sean-k-mooney non admins ususally dont have the acess required to fix the cause of most nova excpetions
16:07:54 luyao sean-k-mooney: sorry, you mean post live migration?
16:08:41 sean-k-mooney luyao: yes we clean up those resouces ealier in the function
16:08:59 sean-k-mooney luyao: so we unplug the guest interface on the ovs bridge for example
16:09:20 sean-k-mooney and we have to unmount any cinder volumes that were mounted on the soucres node
16:09:30 luyao sean-k-mooney: yeah, invoking driver.cleanup will not cleanup them again
16:09:44 openstackgerrit Lee Yarwood proposed openstack/nova master: virt: Provide block_device_info during rescue https://review.opendev.org/700811
16:09:44 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Add support for stable device rescue https://review.opendev.org/700812
16:09:45 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Report COMPUTE_RESCUE_BFV and check during rescue https://review.opendev.org/701429
16:09:45 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Extract _get_bdm_image_metadata into nova.utils https://review.opendev.org/705212
16:09:46 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Support boot from volume stable device instance rescue https://review.opendev.org/701431
16:09:46 openstackgerrit Lee Yarwood proposed openstack/nova master: api: Introduce microverion 2.83 allowing boot from volume rescue https://review.opendev.org/701430
16:09:47 openstackgerrit Lee Yarwood proposed openstack/nova master: DNM - Test stable device rescue tests with BFV instances https://review.opendev.org/710050
16:10:09 sean-k-mooney luyao: ya i know
16:10:22 sean-k-mooney well with the flags you have set
16:10:23 gmann sean-k-mooney: brinzhang and that is what use case if actually. expose something a non-admin could fix. may be filter or whitelist the non-admin fixable exceptions can be better here ?
16:10:42 gmann or at least not expose the non-nova exception at all.
16:11:10 sean-k-mooney gmann: well i would geuss any 4xx errors should be actionable by them in some way
16:11:27 sean-k-mooney if we are identifying them as client issues
16:11:30 luyao sean-k-mooney: yeah, and now I need driver.cleanup to cleanup vpmems
16:12:28 gmann sean-k-mooney: yeah most of them yes. few 404 might not be but I have not checked all exceptions but overall 4xx is in their range
16:17:19 luyao sean-k-mooney, lyarwood: I'll offline and can't response promptly, so please left comments on patch https://review.opendev.org/#/c/687856 if you have any suggestion about vpmem cleanup during live migration. Many Thanks. :)
16:24:57 sean-k-mooney luyao: sure
16:25:03 sean-k-mooney luyao: o/
16:32:20 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add test coverage of existing flavor_manage policies https://review.opendev.org/714814
16:53:31 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reproduce bug 1869050 https://review.opendev.org/714997
16:53:32 openstack bug 1869050 in OpenStack Compute (nova) "migration of anti-affinity server fails due to stale scheduler instance info" [Low,Triaged] https://launchpad.net/bugs/1869050 - Assigned to Balazs Gibizer (balazs-gibizer)
16:53:32 openstackgerrit Balazs Gibizer proposed openstack/nova master: Update scheduler instance info at confirm resize https://review.opendev.org/714998
16:56:37 hrw https://review.opendev.org/#/c/709494 - can someone take a look so aarch64 will be a bit better in nova?
16:58:09 openstackgerrit John Garbutt proposed openstack/nova master: Update quota sets APIs https://review.opendev.org/712749
16:58:09 openstackgerrit John Garbutt proposed openstack/nova master: Tell oslo.limit how to count nova resources https://review.opendev.org/713301
17:06:53 melwitt kashyap: would you mind revisiting the aarch64 patch, it's been updated ^
17:09:32 openstackgerrit Lee Yarwood proposed openstack/nova master: WIP libvirt: Break up get_disk_mapping within blockinfo https://review.opendev.org/714962
17:12:05 sean-k-mooney alex_xu: dansmith gibi so just did a evacuate test with the cyborg fake driver. http://paste.openstack.org/show/791153/
17:12:06 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Update and correct typing information https://review.opendev.org/714694
17:12:06 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Add typing information https://review.opendev.org/714695
17:12:07 openstackgerrit Stephen Finucane proposed openstack/nova master: tests: Split instance NUMA object tests https://review.opendev.org/714696
17:12:07 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: Replace 'cpu_pinning_requested' helper https://review.opendev.org/714697
17:12:08 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Don't consider overhead CPUs for unpinned instances https://review.opendev.org/714698
17:12:08 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Remove handling of pre-Train compute nodes https://review.opendev.org/714699
17:12:09 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Add validation for 'cpu_realtime_mask' https://review.opendev.org/468203
17:12:09 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Tweak the 'cpu_realtime_mask' handling slightly https://review.opendev.org/461456
17:12:10 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Rework 'get_realtime_constraint' https://review.opendev.org/714700
17:12:10 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Invert order of NUMA topology generation https://review.opendev.org/714701
17:12:49 sean-k-mooney alex_xu: dansmith gibi we can evacuate but it does not create allocation for the fpga
17:15:03 sean-k-mooney the arqs are also not updated http://paste.openstack.org/show/791154/
17:15:34 sean-k-mooney ill update the block operation patch review with that info but currently we cannot evacuate properly.
17:16:08 lyarwood stephenfin: https://review.opendev.org/#/c/696834/ - not sure if you're still here but this should be ready now.
17:18:50 stephenfin lyarwood: done
17:19:30 lyarwood stephenfin: thanks
17:20:44 openstackgerrit John Garbutt proposed openstack/nova master: WIP: Enforce resource limits using oslo.limit https://review.opendev.org/615180
17:21:46 openstackgerrit John Garbutt proposed openstack/nova master: Prevent compute manager freeze when greenpool is full https://review.opendev.org/575034
17:27:24 openstackgerrit melanie witt proposed openstack/nova stable/train: Add config option for neutron client retries https://review.opendev.org/715010
17:30:15 openstackgerrit John Garbutt proposed openstack/nova master: Prevent compute manager freeze when greenpool is full https://review.opendev.org/575034
17:31:54 melwitt lyarwood: I dunno if you saw my comment on this one https://review.opendev.org/708030 IIUC this is an option you're thinking to keep indefinitely, if so, it shouldn't go under [workarounds] as they're things intended to be temporary and removed
17:44:03 kashyap melwitt: Hiya; will look at the AArch64 thing tom. in the AM. (Aside: just to keep you posted, I'm off from tomm. evening until 31)
17:44:15 kashyap (s/31/31st-Mar/)
17:45:57 kashyap Actually, looking now
17:46:24 melwitt cool thanks!
17:47:03 kashyap melwitt: Okay, they went with the upstream QEMU AArch64 recomm. of model 'max'. Cool

Earlier   Later