Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-10-08
16:17:09 openstackgerrit Matt Riedemann proposed openstack/nova master: doc: fix formatting in mitigation-for-Intel-MDS-security-flaws https://review.opendev.org/687350
16:18:47 dansmith gibi: (or mriedem) are there any examples of not-instance-related notifications I can copy for image pre-caching?
16:19:03 mriedem there are notifications for aggregates and services
16:19:11 mriedem agg add/remove host i think
16:19:41 dansmith ah yep
16:20:19 mriedem there are also the more generic compute_task.* ones but i'm guessing we should follow the aggregate ones since this is on the aggregates route
16:20:45 dansmith well, the aggregates ones are api-centric, but this would be host-centric
16:20:48 mriedem aggregate.image.cache.start|end?
16:20:51 dansmith i.e. "host foo started downloading a thing"
16:21:01 dansmith no, I want it to be host-specific so you can monitor progress
16:22:03 dansmith although I guess notify_about_aggregate_update could be sent from the conductor
16:22:08 dansmith as long as it mentions the host
16:22:42 dansmith that would allow us to notify about timeouts, etc
16:23:13 mriedem i think notify_about_aggregate_update is the legacy thing
16:23:21 mriedem notify_about_aggregate_action is the versioned notification thing
16:23:44 mriedem there is also a legacy notify_about_host_update for things on the os-hosts api
16:23:48 dansmith okay
16:23:51 mriedem and that's likely closer to what you're looking for
16:24:15 mriedem i don't think those got converted to versioned notifications b/c the os-hosts api is deprecated
16:27:10 openstackgerrit Stephen Finucane proposed openstack/nova master: docs: Add redirects for '/user/aggregates' https://review.opendev.org/687353
16:27:15 stephenfin mriedem: ^
16:27:38 mriedem yar matey
16:30:02 mriedem huh, that patch is triggering functional jobs, must have something missing in our zuul yaml blacklist
16:30:19 dansmith mriedem: so, I'm not really sure what the rules are on our new notifications.. I think I could use AggregatePayload to convey what I want, but not sure if I should
16:30:47 dansmith i.e. AggregatePayload(name="aggregate.imagecache.start", uuid=agg.uuid, hosts=[this_compute])
16:30:59 mriedem no i don't think that's what you want
16:31:52 mriedem my guess is you'll end up creating a new single purpose thing like VolumeUsagePayload and MetricsPayload
16:32:48 mriedem so,
16:32:56 mriedem maybe you're thinking of something like this:
16:33:08 mriedem 1. generic aggregate payload based thing to start the operation sent from conductor
16:33:17 mriedem 2. start/end notifications per compute which would be a new payload
16:33:28 mriedem 3. end version of #1 in conductor once it's done processing all hosts
16:33:58 mriedem so if a consumer cares about only the overall op being done and not the host specific details they can just listen for that
16:34:04 dansmith yeah
16:34:14 mriedem so for 1 and 3 you'd use notify_about_aggregate_action
16:34:17 mriedem with a new action
16:34:19 dansmith so is that a new notification and payload?
16:34:20 mriedem #2 will be a new payload
16:34:44 mriedem notify_about_aggregate_action is what's used for the existing aggregate actions like create/delete/update meta/add and remove host
16:34:51 dansmith meaning, AggregateImageNotification, AggregateImagePayload
16:34:52 sean-k-mooney is the per host version too verbose. not that i think its a bad idea but im wonder would you jsut wait for the conductor start/end
16:34:57 mriedem i think 1 and 3 above fit into that with a new 'cache_images' action or something
16:35:29 dansmith sean-k-mooney: I'm already waiting in the conductor.. I want them to be able to construct the per-host status from notifications if they want, which was part of the original discussion
16:35:38 mriedem i would start with the 1 and 3 cases in conductor since that's pretty trivial
16:35:54 sean-k-mooney dansmith: ah ok
16:35:57 mriedem and then separately bake in the new per-host payload and such
16:35:59 mriedem with gibi's input
16:36:16 dansmith mriedem: yep, separate patch fo'sho
16:36:47 sean-k-mooney dansmith: ya i was wondering if people would go to the effort of reconstructing the hot view but i can see wanting to do that
16:37:04 sean-k-mooney *host
16:37:23 dansmith sean-k-mooney: I want them to be able to so I can use that to deflect requests for a full reporting API, at least initially
16:37:33 mriedem was just going to say that ^
16:37:36 sean-k-mooney :)
16:38:29 mriedem if per host notifications are too noisy for some deployments we can make that configurable - we have an option to include bdms in instance payloads for example
16:41:13 dansmith yeah
16:41:34 dansmith also, it's two notifications per image per compute.. images take a few seconds to download at a minimum in most real cases, so... meh
16:41:44 dansmith we have a bunch per each instance boot already
16:45:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Skip functional test jobs for doc redirect changes https://review.opendev.org/687357
16:46:10 openstackgerrit Matt Riedemann proposed openstack/nova master: docs: Add redirects for '/user/aggregates' https://review.opendev.org/687353
16:54:15 dansmith mriedem: the fake notifier doesn't grab the versioned kind I guess
16:54:23 dansmith but I don't see a fixture or anything for that
16:54:28 efried mriedem: sorry, was otp, reading (https://review.opendev.org/#/c/686835/2/nova/tests/functional/test_boot_from_volume.py@192)
16:54:57 dansmith ah
16:55:01 dansmith nevermind I think
16:56:31 efried mriedem: ack the above, all makes sense to me, thanks for the response.
17:27:38 mriedem dansmith: you found https://github.com/openstack/nova/blob/master/nova/tests/unit/fake_notifier.py#L152 right?
17:27:40 openstackgerrit Dan Smith proposed openstack/nova master: Add cache_images() to conductor https://review.opendev.org/687139
17:27:41 openstackgerrit Dan Smith proposed openstack/nova master: Add image caching API for aggregates https://review.opendev.org/687140
17:27:41 openstackgerrit Dan Smith proposed openstack/nova master: WIP: Add image precaching docs for aggregates https://review.opendev.org/687348
17:27:46 dansmith mriedem: yeah sorry
17:35:10 openstackgerrit Eric Fried proposed openstack/os-traits master: Add method to visualize the os-traits tree https://review.opendev.org/686464
17:39:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Add boot from volume functional test with a huge request https://review.opendev.org/686734
17:39:34 openstackgerrit Matt Riedemann proposed openstack/nova master: Cache image GETs for multi-create/multi-BDM requests https://review.opendev.org/686835
17:44:17 mriedem we might as well revert https://review.opendev.org/684118 since the logging with traceback isn't useful: http://paste.openstack.org/show/782116/
17:45:41 efried boo
17:45:54 efried mriedem: is it any worse than what was before?
17:46:24 mriedem it's not helpful and the warning loses the cell uuid part of the message
17:46:31 mriedem so i think it's not better
17:47:02 efried Oh, I thought the whole point was that the cell UUID was being lost before anyway.
17:47:10 efried but then the unboundlocal thing means we don't have it here either
17:47:12 efried sad face
17:50:24 mriedem we might have something more useful for the db layer if we extended the cell timeout since we might not be getting the root db layer failure because of connection retries in oslo.db
17:50:34 mriedem but i'm still not sure that's going to help debug the issue
17:50:59 mriedem we already know that during grenade we start mysqld 3 times and at some point things go south after the last restart
17:51:36 mriedem we start seeing all of these "Aborted connection" errors
17:51:57 mriedem a la https://review.opendev.org/#/c/682661
17:53:43 mriedem random thinking out loud,
17:53:58 mriedem i wonder if with the instance_extra.resources stuff we've tipped the size of the instance extra queries over
17:56:01 mriedem looking at http://codesearch.openstack.org/?q=max_allowed_packet&i=nope&files=&repos=
17:56:13 mriedem it looks like anywhere from 16M to 256M is pretty standard
18:09:16 openstackgerrit Eric Fried proposed openstack/nova master: Deprecate [api]auth_strategy https://review.opendev.org/687373
18:12:47 mriedem efried: https://review.opendev.org/687374 for the timeout to cell thing like your patch
18:14:33 efried ack, watching
18:15:59 efried mriedem: ...but it didn't work afaict.
18:23:48 openstackgerrit Eric Fried proposed openstack/os-traits master: Privatize and reorganize base package methods https://review.opendev.org/687378
19:33:47 openstackgerrit Merged openstack/nova stable/train: docs: Highlight the current broken state of SEV https://review.opendev.org/686414
19:33:59 openstackgerrit Merged openstack/nova master: nova-net: Make even more nova-net stuff optional https://review.opendev.org/686801
19:36:24 openstackgerrit Artom Lifshitz proposed openstack/nova master: Helper to start computes with different HostInfos https://review.opendev.org/686832
19:36:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
19:36:27 openstackgerrit Artom Lifshitz proposed openstack/nova master: [WIP] NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404

Earlier   Later