| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-01 | |||
| 20:13:14 | mriedem | this is a LIST operation | |
| 20:13:25 | mriedem | let the client sort it out | |
| 20:13:41 | mriedem | let me see how latent this is | |
| 20:15:18 | mriedem | i also wonder if it behaves this way because of https://github.com/openstack/nova/blob/3a2a181796664fea7f6069d49f9cd367b8cff825/nova/compute/api.py#L2362 | |
| 20:15:35 | mriedem | if you filter on fixed_ip it uses an exact regex match | |
| 20:16:09 | mriedem | the tempest test doesn't use that though | |
| 20:18:28 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Release note for cell_name deprecation https://review.openstack.org/439822 | |
| 20:19:12 | mriedem | mtreinish: i put a -1 on the tempest patch and added nova to the bug report | |
| 20:22:05 | mriedem | mtreinish: so it looks like when vish added that long ago it returned all matches https://github.com/openstack/nova/commit/ae781ee97947c33d6d43e4c21df4f338c875bf1c | |
| 20:22:15 | mriedem | https://github.com/openstack/nova/commit/ae781ee97947c33d6d43e4c21df4f338c875bf1c#diff-516904cc81cade24a9122ecf96707bf0R2029 | |
| 20:22:44 | mriedem | this made it short circuit https://github.com/openstack/nova/commit/8fc7eee58ebd929dda75e467f93c381091da7ace | |
| 20:22:53 | erhudy | does anyone have any top tips for taking a particular SQL query and walking back to the SQLAlchemy statements that are responsible for it? perhaps particular settings in default_log_levels would print something useful out? | |
| 20:23:20 | mriedem | erhudy: i think there are trace options in the oslo.db section | |
| 20:24:25 | mtreinish | mriedem: heh, it always comes back to vish | |
| 20:25:56 | mriedem | heh, it's actually kaufer's regression | |
| 20:26:09 | mriedem | don't worry, i'm harranging him internally | |
| 20:26:18 | mriedem | for someone he did 2 years ago and doesn't remember at all | |
| 20:26:21 | mriedem | *something | |
| 20:32:13 | sdague | erhudy: the generated sql is emitted by sqla at debug level, but we turn that off because its *so* noisy. If you play with some of the oslo.log default configs you can tweak it from there | |
| 20:32:37 | erhudy | yeah, i just tried connection_trace and that blew up in my face with a stack trace, connection_debug is pretty noisy but getting closer | |
| 20:32:43 | Gorian | mriedem: sorry, ran away to lunch | |
| 20:32:50 | Gorian | I write python, so I 'm okay with Zen :P | |
| 20:32:57 | erhudy | somebody is running some ansible playbook against a cluster with 25 forks and it is causing the database to go nuclear | |
| 20:33:23 | erhudy | so i am on a spelunking expedition to understand why | |
| 20:33:26 | mriedem | erhudy: ok so you found those, sorry i was looking them up but got distracted | |
| 20:33:33 | mriedem | erhudy: zzzeek might have pointers | |
| 20:34:04 | sdague | erhudy: https://github.com/openstack/oslo.log/blob/5d1a93bc0f597768e27bd116de9d7a66b4cd1f24/oslo_log/_options.py#L20 | |
| 20:34:30 | erhudy | thanks | |
| 20:35:11 | mriedem | sdague: so i guess https://review.openstack.org/#/c/324720/ is happyish | |
| 20:35:17 | mriedem | not sure the xen or vmware failures are related | |
| 20:35:38 | mriedem | oh the vmware one is from november | |
| 20:35:39 | sdague | erhudy: I think that exposes as default_log_levels back in any of the projects, so replace it with the list you like, then debug logs for sqla should come back | |
| 20:35:39 | mriedem | hah | |
| 20:35:59 | sdague | mriedem: I guess it didn't like the recheck | |
| 20:36:36 | sdague | mriedem: yeh, just comment what you would like in a reno (if so), and I'll update the commit message for merge | |
| 20:36:47 | mriedem | the vmware one isn't from a recheck | |
| 20:36:47 | sdague | it pulls out future sec exposures as well | |
| 20:36:58 | sdague | right, but when I update the patch, they should revote | |
| 20:36:59 | mriedem | speaking of vmware....bueller? | |
| 20:37:02 | mriedem | they won't | |
| 20:37:05 | sdague | oh | |
| 20:37:14 | sdague | well, it's a libvirt feature | |
| 20:37:27 | sdague | I'm pretty sure you can't get there from anywhere else | |
| 20:37:29 | mriedem | is it? | |
| 20:39:01 | sdague | git grep inject_data | |
| 20:39:13 | mriedem | the xen ci failure was a stack failure | |
| 20:39:14 | sdague | I see libvirt and xen | |
| 20:39:22 | sdague | but xen is a different function | |
| 20:40:22 | sdague | yeh, xen half end runs around this with their own localfs call | |
| 20:40:29 | sdague | https://github.com/openstack/nova/blob/37cd9a961b065a07352b49ee72394cb210d8838b/nova/virt/xenapi/vm_utils.py#L2479 | |
| 20:41:34 | sdague | this would take it out of the libvirt path | |
| 20:41:45 | mriedem | ok, i assumed you'd be removing VFSLocalFS altogether | |
| 20:41:46 | sdague | the xen path has a fallback here when the xenagent isn't a thing | |
| 20:42:45 | sdague | well this patch was a DNM exploring a security bug | |
| 20:43:18 | sdague | a xen person would have to get on board about what the xen path looks like here, the libvirt one is pretty straight forward | |
| 20:47:25 | openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: remove *_topic config opts https://review.openstack.org/433924 | |
| 21:03:10 | cfriesen | bauzas: it looks like we set request_spec.ignore_hosts in compute.api.API.resize(), so we can't overwrite it later when we call reset_forced_destinations() from nova.conductor.tasks.migrate.MigrationTask.execute() | |
| 21:20:05 | dansmith | dtp: still around? | |
| 21:20:09 | dtp | yessir | |
| 21:20:30 | dansmith | dtp: so I think we're failing to get a list of tests that fail because we're just timing out the job on my patch | |
| 21:20:34 | dansmith | too many fails means too many delays | |
| 21:20:43 | dansmith | I ran the whole deal locally and got a good report | |
| 21:20:57 | dansmith | looking at it, the hypervisors list api is probably a self-contained thing you could work on if you want | |
| 21:21:04 | dansmith | tempest.api.compute.admin.test_hypervisor.HypervisorAdminTestJSON | |
| 21:21:22 | dansmith | if you run that module locally you'll get five tests that fail and then you can go from there | |
| 21:21:38 | dansmith | can probably start with just poking at hypervisors with the client and then run the test once you think it's working | |
| 21:21:41 | dansmith | interested? | |
| 21:22:43 | dtp | sure | |
| 21:22:50 | openstackgerrit | Kaitlin Farr proposed openstack/nova master: Remove deprecated keymgr code https://review.openstack.org/439855 | |
| 21:23:21 | dansmith | cool | |
| 21:23:54 | dtp | i've not done any tempest stuff before. that's external to nova right? | |
| 21:24:23 | dansmith | yeah, you only need to run it though no mods required | |
| 21:24:34 | dansmith | and only after you poke with nova hypervisor-list and think you have it fixed | |
| 21:25:05 | dtp | oh - nova hypervisor-list is having the issue? | |
| 21:26:00 | dansmith | right, and thus tempest is breaking because it's checking that interface | |
| 21:26:23 | dansmith | it's actually not the client having the problem, nor tempest.. it's the server-side api code | |
| 21:26:32 | dtp | i see | |
| 21:26:52 | dansmith | so, nova/api/openstack/compute/hypervisors.py or something | |
| 21:27:01 | dansmith | that api now needs to iterate over cells to gather the list | |
| 21:27:17 | dtp | ah, ok | |
| 21:28:36 | dtp | and which patch of yours? | |
| 21:29:00 | dansmith | this set: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:multi-cell-testing | |
| 21:29:10 | dtp | thanks | |
| 21:29:14 | dansmith | plus this devstack patch which creates things in multiple cells: | |
| 21:29:22 | dansmith | https://review.openstack.org/#/c/436094/ | |
| 21:30:15 | openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: resolved final todos in libvirt conf https://review.openstack.org/429042 | |
| 21:33:06 | mriedem | sdague: left a comment about the reno in your localfs removal change | |
| 21:33:08 | mriedem | with thoughts anyway | |
| 21:33:23 | mriedem | i'm cool with leaving VFSLocalFS in for xenapi to sort out, BobBall should probably take a look | |
| 21:38:23 | dansmith | dtp: this is the tempest module: https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_hypervisor.py | |
| 21:38:26 | dtp | mriedem is there a deadline for pike blueprint submissions (no spec) | |
| 21:38:27 | dansmith | with multiple cells, it fails to list all the hypervisors | |
| 21:38:41 | dtp | ok, thanks dansmith | |
| 21:38:51 | mriedem | dtp: you probably want it done by the first milestone | |
| 21:39:00 | dtp | ok ty | |
| 21:39:02 | dansmith | dtp: fail looks like this: http://pastebin.com/wyh1jEYT | |
| 21:39:48 | dtp | cool, thanks dansmith | |
| 21:39:58 | mriedem | dtp: https://wiki.openstack.org/wiki/Nova/Pike_Release_Schedule#Dates_Overview | |
| 21:40:17 | mriedem | dtp: i was going to bring up spec freeze in the nova meeting tomorrow | |
| 21:40:21 | mriedem | since we didn't talk about that at the ptg | |