Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-10-08
16:31:52 mriedem my guess is you'll end up creating a new single purpose thing like VolumeUsagePayload and MetricsPayload
16:32:48 mriedem so,
16:32:56 mriedem maybe you're thinking of something like this:
16:33:08 mriedem 1. generic aggregate payload based thing to start the operation sent from conductor
16:33:17 mriedem 2. start/end notifications per compute which would be a new payload
16:33:28 mriedem 3. end version of #1 in conductor once it's done processing all hosts
16:33:58 mriedem so if a consumer cares about only the overall op being done and not the host specific details they can just listen for that
16:34:04 dansmith yeah
16:34:14 mriedem so for 1 and 3 you'd use notify_about_aggregate_action
16:34:17 mriedem with a new action
16:34:19 dansmith so is that a new notification and payload?
16:34:20 mriedem #2 will be a new payload
16:34:44 mriedem notify_about_aggregate_action is what's used for the existing aggregate actions like create/delete/update meta/add and remove host
16:34:51 dansmith meaning, AggregateImageNotification, AggregateImagePayload
16:34:52 sean-k-mooney is the per host version too verbose. not that i think its a bad idea but im wonder would you jsut wait for the conductor start/end
16:34:57 mriedem i think 1 and 3 above fit into that with a new 'cache_images' action or something
16:35:29 dansmith sean-k-mooney: I'm already waiting in the conductor.. I want them to be able to construct the per-host status from notifications if they want, which was part of the original discussion
16:35:38 mriedem i would start with the 1 and 3 cases in conductor since that's pretty trivial
16:35:54 sean-k-mooney dansmith: ah ok
16:35:57 mriedem and then separately bake in the new per-host payload and such
16:35:59 mriedem with gibi's input
16:36:16 dansmith mriedem: yep, separate patch fo'sho
16:36:47 sean-k-mooney dansmith: ya i was wondering if people would go to the effort of reconstructing the hot view but i can see wanting to do that
16:37:04 sean-k-mooney *host
16:37:23 dansmith sean-k-mooney: I want them to be able to so I can use that to deflect requests for a full reporting API, at least initially
16:37:33 mriedem was just going to say that ^
16:37:36 sean-k-mooney :)
16:38:29 mriedem if per host notifications are too noisy for some deployments we can make that configurable - we have an option to include bdms in instance payloads for example
16:41:13 dansmith yeah
16:41:34 dansmith also, it's two notifications per image per compute.. images take a few seconds to download at a minimum in most real cases, so... meh
16:41:44 dansmith we have a bunch per each instance boot already
16:45:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Skip functional test jobs for doc redirect changes https://review.opendev.org/687357
16:46:10 openstackgerrit Matt Riedemann proposed openstack/nova master: docs: Add redirects for '/user/aggregates' https://review.opendev.org/687353
16:54:15 dansmith mriedem: the fake notifier doesn't grab the versioned kind I guess
16:54:23 dansmith but I don't see a fixture or anything for that
16:54:28 efried mriedem: sorry, was otp, reading (https://review.opendev.org/#/c/686835/2/nova/tests/functional/test_boot_from_volume.py@192)
16:54:57 dansmith ah
16:55:01 dansmith nevermind I think
16:56:31 efried mriedem: ack the above, all makes sense to me, thanks for the response.
17:27:38 mriedem dansmith: you found https://github.com/openstack/nova/blob/master/nova/tests/unit/fake_notifier.py#L152 right?
17:27:40 openstackgerrit Dan Smith proposed openstack/nova master: Add cache_images() to conductor https://review.opendev.org/687139
17:27:41 openstackgerrit Dan Smith proposed openstack/nova master: Add image caching API for aggregates https://review.opendev.org/687140
17:27:41 openstackgerrit Dan Smith proposed openstack/nova master: WIP: Add image precaching docs for aggregates https://review.opendev.org/687348
17:27:46 dansmith mriedem: yeah sorry
17:35:10 openstackgerrit Eric Fried proposed openstack/os-traits master: Add method to visualize the os-traits tree https://review.opendev.org/686464
17:39:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Add boot from volume functional test with a huge request https://review.opendev.org/686734
17:39:34 openstackgerrit Matt Riedemann proposed openstack/nova master: Cache image GETs for multi-create/multi-BDM requests https://review.opendev.org/686835
17:44:17 mriedem we might as well revert https://review.opendev.org/684118 since the logging with traceback isn't useful: http://paste.openstack.org/show/782116/
17:45:41 efried boo
17:45:54 efried mriedem: is it any worse than what was before?
17:46:24 mriedem it's not helpful and the warning loses the cell uuid part of the message
17:46:31 mriedem so i think it's not better
17:47:02 efried Oh, I thought the whole point was that the cell UUID was being lost before anyway.
17:47:10 efried but then the unboundlocal thing means we don't have it here either
17:47:12 efried sad face
17:50:24 mriedem we might have something more useful for the db layer if we extended the cell timeout since we might not be getting the root db layer failure because of connection retries in oslo.db
17:50:34 mriedem but i'm still not sure that's going to help debug the issue
17:50:59 mriedem we already know that during grenade we start mysqld 3 times and at some point things go south after the last restart
17:51:36 mriedem we start seeing all of these "Aborted connection" errors
17:51:57 mriedem a la https://review.opendev.org/#/c/682661
17:53:43 mriedem random thinking out loud,
17:53:58 mriedem i wonder if with the instance_extra.resources stuff we've tipped the size of the instance extra queries over
17:56:01 mriedem looking at http://codesearch.openstack.org/?q=max_allowed_packet&i=nope&files=&repos=
17:56:13 mriedem it looks like anywhere from 16M to 256M is pretty standard
18:09:16 openstackgerrit Eric Fried proposed openstack/nova master: Deprecate [api]auth_strategy https://review.opendev.org/687373
18:12:47 mriedem efried: https://review.opendev.org/687374 for the timeout to cell thing like your patch
18:14:33 efried ack, watching
18:15:59 efried mriedem: ...but it didn't work afaict.
18:23:48 openstackgerrit Eric Fried proposed openstack/os-traits master: Privatize and reorganize base package methods https://review.opendev.org/687378
19:33:47 openstackgerrit Merged openstack/nova stable/train: docs: Highlight the current broken state of SEV https://review.opendev.org/686414
19:33:59 openstackgerrit Merged openstack/nova master: nova-net: Make even more nova-net stuff optional https://review.opendev.org/686801
19:36:24 openstackgerrit Artom Lifshitz proposed openstack/nova master: Helper to start computes with different HostInfos https://review.opendev.org/686832
19:36:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
19:36:27 openstackgerrit Artom Lifshitz proposed openstack/nova master: [WIP] NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404
19:36:27 openstack bug 1845146 in OpenStack Compute (nova) train "NUMA aware live migration failed when vCPU pin set" [High,Fix committed] https://launchpad.net/bugs/1845146 - Assigned to Dan Smith (danms)
20:18:33 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Bulk query floating IPs in _nw_info_get_ips https://review.opendev.org/465792
20:18:34 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: Time how long _nw_info_get_ips takes https://review.opendev.org/687410
20:31:07 openstackgerrit Merged openstack/nova master: doc: fix formatting in mitigation-for-Intel-MDS-security-flaws https://review.opendev.org/687350
20:33:03 openstackgerrit Eric Fried proposed openstack/nova master: Deprecate [api]auth_strategy and noauth2 https://review.opendev.org/687373
20:33:04 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Murder NoAuthMiddleware https://review.opendev.org/687416
20:33:10 openstackgerrit Matt Riedemann proposed openstack/nova master: doc: link to user/availability-zones from user home page https://review.opendev.org/687165
20:33:10 openstackgerrit Matt Riedemann proposed openstack/nova master: doc: link to user/index from main home page https://review.opendev.org/687292
20:33:41 efried mriedem: It turned out to be actually quite easy to bypass noauth and just stub out keystone --^
20:34:04 efried there's a lot more to be done there, but I was able to get through enough pieces to convince myself that that approach is going to work.
20:34:23 efried the remainder is just cleaning up the leftovers.
20:34:55 mriedem does ksa/keystonemiddleware provide any fixtures for some of this?
20:35:15 mriedem that would be ideal rather than stubbing keystonemiddleware.auth_token.filter_factory
20:35:35 mriedem oh hello https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/fixture.py
20:35:43 efried mriedem: I agree, but that would be a subsequent step in any case.
20:36:18 efried as in, doing this is not taking away anything we were doing before, because we weren't hitting those factories at all
20:36:34 efried I mentioned same in my commit message
20:36:59 efried lbragstad: any idea if there's a fixture for keystonemiddleware.auth_token.filter_factory ?
20:38:01 efried ah, neat
20:38:13 efried lbragstad: cancel, I did not read scrollback thoroughly.
20:38:32 lbragstad yup - looks like you're on the right track
20:40:23 mriedem efried: ultra mega -1 on the bottom change
20:40:36 mriedem not the logging thing, the reno
20:41:04 efried mriedem: o ya, I thought of that while I was updating and then forgot again, will do.
20:42:42 mriedem so with https://review.opendev.org/#/c/686414/ i guess we can push rc2 now yeah?
20:42:44 mriedem are you doing that?

Earlier   Later