| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-22 | |||
| 17:23:36 | dansmith | did we ever log them? | |
| 17:23:52 | dansmith | they'd be spread all over the place if we did | |
| 17:24:12 | melwitt | based on the existence of the CONF.notifications.default_level option we did | |
| 17:24:13 | melwitt | yeah | |
| 17:24:26 | melwitt | but the option is no longer used anywhere after that commit | |
| 17:24:42 | melwitt | so I wonder if that decorator used to result in logging somehow. I don't know how it worked | |
| 17:25:25 | dansmith | isn't that level something about which notifcations we emit, not the log level? | |
| 17:26:13 | dansmith | "Default notification level for outgoing notifications." | |
| 17:26:18 | melwitt | I dunno, sounds like a log level https://github.com/openstack/nova/blob/master/nova/conf/notifications.py#L51-L57 | |
| 17:27:05 | melwitt | but yeah, I guess it could have meant that notifications used to be categorized and you could prune down which to emit? | |
| 17:27:23 | dansmith | right, like only emit started/stopped type things and not progress-related ones | |
| 17:27:32 | dansmith | or "I checked on a thing" ones | |
| 17:27:36 | dansmith | I could definitely be wrong | |
| 17:27:44 | melwitt | maybe we never logged notifications. guh, I don't know | |
| 17:27:44 | sean-k-mooney | melwitt: we have some failutre logs here https://github.com/openstack/nova/blob/master/nova/notifications/base.py#L134-L139 | |
| 17:27:54 | dansmith | I didn't really think we did, tbh | |
| 17:28:06 | melwitt | sean-k-mooney: yeah, I know we have failure logs | |
| 17:40:57 | mriedem | melwitt: we do'nt log notificatoins | |
| 17:41:29 | mriedem | like dan said, that's not a log level | |
| 17:41:36 | mriedem | each event has a level associated with it | |
| 17:42:02 | mriedem | why? what customer is saying they had notifications logged? | |
| 17:42:12 | mriedem | note - you can configure oslo.messaging to just log notifications | |
| 17:42:17 | mriedem | that's not a nova thing, that's an oslo thing | |
| 17:42:25 | melwitt | thanks. fyi, the default_level option is no longer used anywhere so that might be a bug? but unrelated to what I was looking for | |
| 17:43:01 | mriedem | https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html#oslo_messaging_notifications.driver | |
| 17:43:09 | mriedem | [oslo_messaging_notifications]/driver=log | |
| 17:43:15 | melwitt | mriedem: no, we're trying to debug something and trying to verify that nova is sending notifications. and were looking for a clue about how to tell if it is | |
| 17:43:17 | melwitt | thanks | |
| 17:44:15 | mriedem | btw, gibi's last summit talk about versioned notifications (which i think you're in the video) has some nice stuff about configuring and testing notificaoins out to see them | |
| 17:44:42 | mriedem | https://www.openstack.org/videos/summits/denver-2019/nova-versioned-notifications-the-result-of-a-3-year-journey | |
| 17:45:11 | mriedem | oh let me guess - something something heat isn't getting notifications | |
| 17:45:43 | mriedem | maybe the customer flipped on versioned notifications only and heat only listens for unversioned | |
| 17:45:48 | melwitt | novajoin, the vendordata service | |
| 17:45:49 | melwitt | so yeah, pretty much | |
| 17:46:15 | melwitt | yeah, we checked and both are set to use unversioned so it's not something obvious | |
| 17:47:48 | melwitt | haha, indeed I attended that talk. I'll look at that again | |
| 17:51:04 | melwitt | looks like something is afoot with the driver setting, it's set to 'noop' which seems bad :( | |
| 17:52:50 | melwitt | good thing you pointed that out, I had forgotten about it | |
| 17:52:57 | melwitt | [oslo_messaging_notifications]/driver | |
| 17:59:23 | efried | sean-k-mooney: were you one who was vocal about what "privsep stage 3" should look like? | |
| 18:03:37 | efried | nm, I see you're already on the cyborg privsep review, which is where this was going. | |
| 18:16:25 | melwitt | looks like there was a recent change to tripleo to disable notification drivers when telemetry is disabled, so this deploy got caught up in that | |
| 18:17:29 | melwitt | thanks again for the hint about the [oslo_messaging_notifications]/driver setting. mriedem++ | |
| 18:23:15 | mriedem | worked? | |
| 18:24:05 | melwitt | they haven't tried yet with driver=messagingv2 but I'm sure this is the issue | |
| 18:24:59 | mriedem | ah yeah | |
| 18:25:59 | melwitt | but novajoin is in the mix here too and needs the notifications to work | |
| 18:25:59 | melwitt | there was an assumption that only telemetry needed the notifications and since it's disabled, it set driver=noop to avoid needless notification traffic, | |
| 18:28:18 | mriedem | silly tripleo | |
| 18:28:30 | mriedem | i thought heat also consumed nova notifications, or maybe that's indirectly via ceilometer? | |
| 18:28:57 | mriedem | but yeah lots of projects listen to nova notifications besides those, you've also got designate, watcher, blazar, etc - though maybe tripleo doesn't support those | |
| 18:29:09 | melwitt | yeah, this is the undercloud and that confuses me, but maybe that's why | |
| 18:29:32 | melwitt | I think notifications are always on in the overcloud | |
| 18:29:59 | melwitt | in order for other things to work, I assume we have some other service that needs them | |
| 18:30:02 | mriedem | oh yeah, forgot about the cloud turducken | |
| 18:30:09 | melwitt | yeah | |
| 18:30:36 | mriedem | well i suppose it always feels good to come back and say, "this isn't a nova problem, it's a tripleo problem" | |
| 18:31:03 | melwitt | heh, indeed | |
| 18:31:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove legacy '_from_dict' functions https://review.opendev.org/537414 | |
| 18:31:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | |
| 18:31:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add '[compute] cpu_dedicated_set' option https://review.opendev.org/671792 | |
| 18:31:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395 | |
| 18:31:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693 | |
| 18:31:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103 | |
| 18:31:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 18:31:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples https://review.opendev.org/675338 | |
| 18:31:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 18:31:18 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu' https://review.opendev.org/676729 | |
| 18:31:18 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 18:31:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 18:38:53 | openstackgerrit | Eric Fried proposed openstack/nova master: Add docstring to check_availability_zone function https://review.opendev.org/677858 | |
| 18:40:55 | mriedem | anyone else have an opinion about nova migration-list with --migration-type or just --type in https://review.opendev.org/#/c/675117/ ? | |
| 18:59:12 | efried | mriedem: sans click, I would say --migration-type just because --type is so generic. | |
| 19:02:45 | artom | mriedem, make it --kind-of-migration and watch the comments pour in | |
| 19:04:19 | mriedem | --migration-flavour? | |
| 19:05:51 | artom | Points for the UK spelling :D | |
| 19:06:13 | artom | (aka the *correct* spelling. Yours truly, a Canadian) | |
| 19:06:48 | artom | (except when I feel like being Quebecois or Russian - yay absence of national identity!) | |
| 19:14:15 | efried | mriedem: I left helpful comments on https://review.opendev.org/#/c/678051/ | |
| 19:20:07 | mriedem | artom: how would you do it in old russ? | |
| 19:20:38 | artom | Gather 'round a bottle of vodka (or two) and let the solution emerge by itself | |
| 19:22:42 | mriedem | efried: you might need: | |
| 19:22:44 | mriedem | become: yes | |
| 19:24:18 | artom | mriedem, FWIW, I'd say just --type, because 'migration' is already explicitly implied (ha!) in the command | |
| 19:24:22 | mriedem | the devstack tasks that use synchronize all have some form of "become: true" for sudo | |
| 19:24:56 | artom | In (server-)list and flavor-list, we don't repeat the thing we're list in the args | |
| 19:25:12 | artom | So it's just flavor-list --extra-specs, not flavor-list --flavor-extra-specs | |
| 19:25:24 | melwitt | I don't know if it's just me but I didn't understand which option dtroyer was for in his comment | |
| 19:25:38 | artom | But I agree that --type is trickier, because it's more vague on its own than something like --extra-specs | |
| 19:29:48 | mriedem | melwitt: dean was saying since nova uses --*-type as a convention be consistent in nova cli | |
| 19:29:54 | mriedem | osc uses --type for the most part | |
| 19:30:03 | mriedem | since context is derived from the command you're using | |
| 19:30:15 | melwitt | does a nova osc command count as 'nova cli'? | |
| 19:30:21 | mriedem | so since nova cli uses --key-type, --console-type and --backup-type, use --migration-type | |
| 19:30:26 | mriedem | no | |
| 19:30:28 | mriedem | osc is osc | |
| 19:30:39 | melwitt | haha ok, that's what I was thinking | |
| 19:37:30 | melwitt | oh, geez, I didn't realize that change was for python-novaclient. ugh | |
| 19:37:50 | melwitt | I thought it was for osc 'openstack server' command | |
| 19:38:48 | melwitt | yeah I agree stay consistent with other python-novaclient things | |