| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-22 | |||
| 17:44:15 | mriedem | btw, gibi's last summit talk about versioned notifications (which i think you're in the video) has some nice stuff about configuring and testing notificaoins out to see them | |
| 17:44:42 | mriedem | https://www.openstack.org/videos/summits/denver-2019/nova-versioned-notifications-the-result-of-a-3-year-journey | |
| 17:45:11 | mriedem | oh let me guess - something something heat isn't getting notifications | |
| 17:45:43 | mriedem | maybe the customer flipped on versioned notifications only and heat only listens for unversioned | |
| 17:45:48 | melwitt | novajoin, the vendordata service | |
| 17:45:49 | melwitt | so yeah, pretty much | |
| 17:46:15 | melwitt | yeah, we checked and both are set to use unversioned so it's not something obvious | |
| 17:47:48 | melwitt | haha, indeed I attended that talk. I'll look at that again | |
| 17:51:04 | melwitt | looks like something is afoot with the driver setting, it's set to 'noop' which seems bad :( | |
| 17:52:50 | melwitt | good thing you pointed that out, I had forgotten about it | |
| 17:52:57 | melwitt | [oslo_messaging_notifications]/driver | |
| 17:59:23 | efried | sean-k-mooney: were you one who was vocal about what "privsep stage 3" should look like? | |
| 18:03:37 | efried | nm, I see you're already on the cyborg privsep review, which is where this was going. | |
| 18:16:25 | melwitt | looks like there was a recent change to tripleo to disable notification drivers when telemetry is disabled, so this deploy got caught up in that | |
| 18:17:29 | melwitt | thanks again for the hint about the [oslo_messaging_notifications]/driver setting. mriedem++ | |
| 18:23:15 | mriedem | worked? | |
| 18:24:05 | melwitt | they haven't tried yet with driver=messagingv2 but I'm sure this is the issue | |
| 18:24:59 | mriedem | ah yeah | |
| 18:25:59 | melwitt | but novajoin is in the mix here too and needs the notifications to work | |
| 18:25:59 | melwitt | there was an assumption that only telemetry needed the notifications and since it's disabled, it set driver=noop to avoid needless notification traffic, | |
| 18:28:18 | mriedem | silly tripleo | |
| 18:28:30 | mriedem | i thought heat also consumed nova notifications, or maybe that's indirectly via ceilometer? | |
| 18:28:57 | mriedem | but yeah lots of projects listen to nova notifications besides those, you've also got designate, watcher, blazar, etc - though maybe tripleo doesn't support those | |
| 18:29:09 | melwitt | yeah, this is the undercloud and that confuses me, but maybe that's why | |
| 18:29:32 | melwitt | I think notifications are always on in the overcloud | |
| 18:29:59 | melwitt | in order for other things to work, I assume we have some other service that needs them | |
| 18:30:02 | mriedem | oh yeah, forgot about the cloud turducken | |
| 18:30:09 | melwitt | yeah | |
| 18:30:36 | mriedem | well i suppose it always feels good to come back and say, "this isn't a nova problem, it's a tripleo problem" | |
| 18:31:03 | melwitt | heh, indeed | |
| 18:31:13 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove legacy '_from_dict' functions https://review.opendev.org/537414 | |
| 18:31:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | |
| 18:31:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add '[compute] cpu_dedicated_set' option https://review.opendev.org/671792 | |
| 18:31:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395 | |
| 18:31:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693 | |
| 18:31:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103 | |
| 18:31:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 18:31:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples https://review.opendev.org/675338 | |
| 18:31:17 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 18:31:18 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu' https://review.opendev.org/676729 | |
| 18:31:18 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 18:31:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 18:38:53 | openstackgerrit | Eric Fried proposed openstack/nova master: Add docstring to check_availability_zone function https://review.opendev.org/677858 | |
| 18:40:55 | mriedem | anyone else have an opinion about nova migration-list with --migration-type or just --type in https://review.opendev.org/#/c/675117/ ? | |
| 18:59:12 | efried | mriedem: sans click, I would say --migration-type just because --type is so generic. | |
| 19:02:45 | artom | mriedem, make it --kind-of-migration and watch the comments pour in | |
| 19:04:19 | mriedem | --migration-flavour? | |
| 19:05:51 | artom | Points for the UK spelling :D | |
| 19:06:13 | artom | (aka the *correct* spelling. Yours truly, a Canadian) | |
| 19:06:48 | artom | (except when I feel like being Quebecois or Russian - yay absence of national identity!) | |
| 19:14:15 | efried | mriedem: I left helpful comments on https://review.opendev.org/#/c/678051/ | |
| 19:20:07 | mriedem | artom: how would you do it in old russ? | |
| 19:20:38 | artom | Gather 'round a bottle of vodka (or two) and let the solution emerge by itself | |
| 19:22:42 | mriedem | efried: you might need: | |
| 19:22:44 | mriedem | become: yes | |
| 19:24:18 | artom | mriedem, FWIW, I'd say just --type, because 'migration' is already explicitly implied (ha!) in the command | |
| 19:24:22 | mriedem | the devstack tasks that use synchronize all have some form of "become: true" for sudo | |
| 19:24:56 | artom | In (server-)list and flavor-list, we don't repeat the thing we're list in the args | |
| 19:25:12 | artom | So it's just flavor-list --extra-specs, not flavor-list --flavor-extra-specs | |
| 19:25:24 | melwitt | I don't know if it's just me but I didn't understand which option dtroyer was for in his comment | |
| 19:25:38 | artom | But I agree that --type is trickier, because it's more vague on its own than something like --extra-specs | |
| 19:29:48 | mriedem | melwitt: dean was saying since nova uses --*-type as a convention be consistent in nova cli | |
| 19:29:54 | mriedem | osc uses --type for the most part | |
| 19:30:03 | mriedem | since context is derived from the command you're using | |
| 19:30:15 | melwitt | does a nova osc command count as 'nova cli'? | |
| 19:30:21 | mriedem | so since nova cli uses --key-type, --console-type and --backup-type, use --migration-type | |
| 19:30:26 | mriedem | no | |
| 19:30:28 | mriedem | osc is osc | |
| 19:30:39 | melwitt | haha ok, that's what I was thinking | |
| 19:37:30 | melwitt | oh, geez, I didn't realize that change was for python-novaclient. ugh | |
| 19:37:50 | melwitt | I thought it was for osc 'openstack server' command | |
| 19:38:48 | melwitt | yeah I agree stay consistent with other python-novaclient things | |
| 19:39:21 | efried | mriedem: it looks like logging is enabled by default? | |
| 19:41:29 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Try to repro bug 1823251 with mysql logs https://review.opendev.org/678051 | |
| 19:41:30 | openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] https://launchpad.net/bugs/1823251 | |
| 19:41:54 | mnaser | sean-k-mooney: do you happen to know if you're able to backport the os-vif bug to stable/stein ? | |
| 19:42:14 | efried | mriedem: if it's writing to "the console"... is that the same console that tests are being run in? | |
| 19:44:03 | mriedem | efried: no, | |
| 19:44:11 | mriedem | likely the journal? | |
| 19:44:19 | mriedem | i.e. mysql systemd journal log | |
| 19:44:34 | mriedem | if i had a devstack handy i'd poke at this but i don't | |
| 19:45:00 | mriedem | efried: if this were an IMM or VIOS, wwyd?! | |
| 19:45:14 | efried | mriedem: How destructive is the poking? I have one you can jump on | |
| 19:46:06 | openstackgerrit | Jonathan Rosser proposed openstack/os-vif stable/stein: only disable mac ageing for ovs hybrid plug https://review.opendev.org/678098 | |
| 19:47:33 | efried | The problem is that I have basically no understanding of the chain of events that's leading to mysqld running and/or being used in this environment. | |
| 19:47:33 | efried | I don't know what IMM is. If it were VIOS I would phone a friend. | |
| 19:48:38 | mnaser | sean-k-mooney: jrosser has backported the os-vif issue which fixes deployments letting vms see other vms traffic :) | |
| 19:49:53 | mriedem | integrated management module | |
| 19:49:58 | mriedem | like the CMM in a bladecenter | |
| 19:50:00 | mriedem | DOY | |
| 19:50:25 | mriedem | where is dansmith when i need him for ibm jokes | |
| 19:51:16 | mriedem | efried: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/test-setup is the role for hitting the test-setup script in the unit test jobs | |
| 19:51:23 | efried | by the time we run tools/test-setup.sh, mysqld is already running, right? | |
| 19:51:52 | mriedem | has to be | |
| 19:51:56 | mriedem | since we're issuing commands against it | |
| 19:52:05 | efried | so whatever we're doing we need to do well before that | |
| 19:52:29 | efried | unless it's possible to restart mysqld from that script, is that what you were suggesting above? | |
| 19:56:29 | melwitt | mnaser: isn't that a pretty serious security issue? looked in the bug and it didn't look treated as such | |
| 19:57:33 | openstackgerrit | Merged openstack/nova master: Remove deprecated [neutron]/url option https://review.opendev.org/677755 | |
| 20:01:04 | mriedem | efried: the mysql docs seem to suggest you can change logging config at runtime and flush the settings | |