| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-01 | |||
| 12:41:57 | kashyap | lyarwood: You only test with RPM-based setups? | |
| 12:43:01 | lyarwood | kashyap: yeah, almost all of the multipath issues I've had to work on recently are for Kilo envs so the volume code doesn't even apply to upstream anymore | |
| 12:43:21 | lyarwood | kashyap: well, not since liberty and the split into os-brick | |
| 12:43:31 | kashyap | lyarwood: Hmm, okay. Yeah, it's just a pity that it's such a pain in the neck to figure out the right configs to even test with upstream | |
| 12:44:02 | kashyap | Seems like nobody bothers with it much | |
| 13:02:53 | openstackgerrit | Stephen Finucane proposed openstack/nova: console: Move proxies to 'console/proxy' https://review.openstack.org/408192 | |
| 13:08:33 | wiggin15 | Hi everyone. Something changed in Nova that fails the Cinder third party CI systems. We're getting "No hosts found to map to cell, exiting.". It started yesterday | |
| 13:08:35 | openstackgerrit | Stephen Finucane proposed openstack/nova: tests: Validate huge pages https://review.openstack.org/399653 | |
| 13:08:36 | openstackgerrit | Stephen Finucane proposed openstack/nova: tests: Combine multiple NUMA-generation functions https://review.openstack.org/402561 | |
| 13:08:36 | openstackgerrit | Stephen Finucane proposed openstack/nova: tests: Don't reinvent __init__ https://review.openstack.org/402560 | |
| 13:08:36 | wiggin15 | Here's Netapp's log: http://dcf901611175aa43f968-c54047c910227e27e1d6f03bb1796fd7.r95.cf5.rackcdn.com/56/427356/1/check/cinder-7mode-iSCSI/0699f1b/devstacklog.txt | |
| 13:08:37 | openstackgerrit | Stephen Finucane proposed openstack/nova: libvirt: create functional test base class https://review.openstack.org/407055 | |
| 13:08:37 | openstackgerrit | Stephen Finucane proposed openstack/nova: tests: Remove duplicate NumaHostInfo https://review.openstack.org/402562 | |
| 13:08:52 | wiggin15 | Here's virtuozzo: http://openstack-3rd-party-virtuozzo-ci-logs.virtuozzo.com/71/416371/9/check/check-dsvm-tempest-vz7-exe-minimal/0854760/logs/devstacklog.txt.gz | |
| 13:08:56 | wiggin15 | others are failing too | |
| 13:09:30 | wiggin15 | Does anyone know if something changed in the past 24 hours that can cause this? | |
| 13:10:34 | cdent | wiggin15: sounds like it could be something related to cellsv2 stuff that recently merged | |
| 13:11:48 | cdent | wiggin15: there was some discussion about some changes that need to happen, last night, but I'm not sure of the outcome or timing. when mriedem comes around he'll probably know more. but also check the os-dev list for messages from him, he's being sending gate updates there | |
| 13:16:22 | macsz | was the Nova API meeting canceled for today or is my calendar messed up? | |
| 13:17:09 | openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/nova: WIP: placement-api: fix ResourceProviderList query https://review.openstack.org/427667 | |
| 13:20:10 | johnthetubaguy | macsz: it was cancelled today | |
| 13:20:54 | macsz | johnthetubaguy: ah, ok | |
| 13:28:09 | cdent | rpodolyaka: on group by thing, are you saying: "we're not using c.consumer_id, so just leave it out of the select"? | |
| 13:28:52 | rpodolyaka | cdent: yeah, my understanding is that we calculate total usage values per RP, not "per RP per consumer" | |
| 13:29:08 | rpodolyaka | and this total actually includes multiple consumers | |
| 13:29:28 | rpodolyaka | so consumer_id in select does not make sense and that's what postgresql is unhappy about | |
| 13:30:14 | cdent | makes sense | |
| 13:30:18 | smcginnis | wiggin15: Sounds like you, like me, ignored this in the ML and now has stacking failures: http://lists.openstack.org/pipermail/openstack-dev/2017-January/111295.html | |
| 13:30:33 | rpodolyaka | we *could* group by consumer_id, but my understanding is that it's not what we really want | |
| 13:30:48 | cdent | rpodolyaka: I think you're right | |
| 13:31:03 | rpodolyaka | but I'm not really familiar with this code, so jaypipes should know better :P | |
| 13:31:32 | cdent | rpodolyaka: but the second group_by change is probably right? | |
| 13:31:36 | cdent | (for postgres?) | |
| 13:32:55 | cdent | sileht: you want to carry on with that stuff. If not I can take it over. | |
| 13:33:17 | openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/nova: placement-api: fix ResourceProviderList query https://review.openstack.org/427667 | |
| 13:33:38 | sileht | rpodolyaka, the functional tests have catched your thought :) | |
| 13:35:31 | rpodolyaka | sileht: yay! :) | |
| 13:36:42 | openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/nova: placement-api: fix ResourceProviderList query https://review.openstack.org/427667 | |
| 13:36:44 | cdent | sileht: being very nit picky on the review in the hope that by the time some core comes along it is nice and happy and we get this merged quick | |
| 13:36:56 | sileht | cdent, | |
| 13:37:11 | sileht | cdent, it's not really nit picky when feedback are instant :) | |
| 13:37:23 | rpodolyaka | cdent: hmm, the second group by is done on a unique column - it must be ok, as it uniquely identifies a particular table row. I'd expect "select rp.* ..." work fine in this case | |
| 13:37:58 | rpodolyaka | tbh it's the first time I've seen a sqalclhemy model passed to group_by() | |
| 13:38:17 | rpodolyaka | my understanding is that it's equivalent of grouping by all columns of a table | |
| 13:38:19 | sileht | rpodolyaka, with postgresql you must have everything you put in the select | |
| 13:38:35 | sileht | rpodolyaka, yes because the select does 'select rp.*' | |
| 13:38:52 | sileht | rpodolyaka, so even if rp.uuid is enought, postgresql don't want to guess | |
| 13:40:01 | sileht | rpodolyaka, I think the second groupby is here just to remove duplicate rows ? | |
| 13:40:22 | mriedem | wiggin15: you need to have the placement-api service enabled, in ENABLED_SERVICES | |
| 13:40:36 | mriedem | wiggin15: if you're overriding ENABLED_SERVICES that is - there is a note in the dev list about it | |
| 13:41:53 | sileht | rpodolyaka, the orignal error we have: http://logs.openstack.org/82/405682/8/check/gate-ceilometer-dsvm-tempest-plugin-postgresql-ubuntu-xenial/02f896f/logs/apache/placement-api.txt.gz?level=ERROR | |
| 13:42:10 | sileht | rpodolyaka, the error when I have fixed the first groupby: http://logs.openstack.org/68/427668/1/check/gate-ceilometer-dsvm-tempest-plugin-postgresql-ubuntu-xenial/372f489/logs/apache/placement-api.txt.gz?level=ERROR | |
| 13:42:22 | mriedem | johnthetubaguy: sfinucan: your call on https://review.openstack.org/#/c/412634/ since it's disabled by default an experimental now anyway | |
| 13:42:36 | mriedem | it was already approved once, just had a busted unit test | |
| 13:43:05 | rpodolyaka | sileht: hmm, interesting | |
| 13:43:08 | cdent | mriedem: I tagged this https://bugs.launchpad.net/nova/+bug/1660959 as rc-potential because postgres support used to work in placement api. it's what sileht and rpodolyaka are talking over now. | |
| 13:43:08 | openstack | Launchpad bug 1660959 in OpenStack Compute (nova) "placement resource provider filtering does not work with postgres" [Undecided,In progress] - Assigned to Mehdi Abaakouk (sileht) | |
| 13:43:15 | mriedem | cdent: yup saw it, thanks | |
| 13:43:17 | mriedem | silly pg | |
| 13:43:24 | wiggin15 | mriedem we have "export OVERRIDE_ENABLED_SERVICES" with services like "c-api", "q-metering" etc., is this it? | |
| 13:43:32 | mriedem | wiggin15: i think so yes | |
| 13:43:35 | mriedem | you need placement-api in that list | |
| 13:43:38 | wiggin15 | mriedem thanks | |
| 13:43:40 | sfinucan | mriedem: I'm happy to let it in, in that case. It's technically a feature but it seems low risk to me. Can't speak for johnthetubaguy, mind | |
| 13:43:40 | sileht | cdent, and our (future) stable/ocata is already broken too :p | |
| 13:43:54 | mriedem | i'm heading into the office, bbiab | |
| 13:44:23 | cdent | sileht: I think the whole world's future is broken lately. | |
| 13:46:29 | rpodolyaka | sileht: heh, looks like it only allows this for PKs, but not for UNIQUEs - http://paste.openstack.org/show/597180/ | |
| 13:48:53 | sileht | rpodolyaka, do we have another column that can do it (instead of putting all cols) ? | |
| 13:51:16 | sileht | rpodolyaka, I will use id, then | |
| 13:51:23 | rpodolyaka | sileht: yeah, we have the PK - `id` | |
| 13:54:02 | openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/nova: placement-api: fix ResourceProviderList query https://review.openstack.org/427667 | |
| 13:56:46 | openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/nova: placement-api: fix ResourceProviderList query https://review.openstack.org/427667 | |
| 14:07:27 | johnthetubaguy | BobBall: seeing some XenAPI issues on this glance related patch: https://review.openstack.org/#/c/412634/24 | |
| 14:07:36 | johnthetubaguy | BobBall: is this a general problem we are seeing at the moment? | |
| 14:08:11 | BobBall | Checking | |
| 14:08:42 | BobBall | general problem meaning unrelated to this patch? | |
| 14:09:08 | johnthetubaguy | BobBall: it feels very related to this patch, but we haven't changed the code that XenAPI uses, AFAIK | |
| 14:09:19 | johnthetubaguy | BobBall: just wanted to check its not a general issue | |
| 14:09:35 | BobBall | hmmmz - actually we are failing all tests in the last couple of hours... | |
| 14:10:40 | johnthetubaguy | BobBall: looks like the plugins have changed or something? | |
| 14:10:52 | BobBall | Big sadness from the Citrix CI at the moment :( | |
| 14:11:45 | BobBall | Investigating. Plugins certainly shouldn't have changed since we're pulling in from a fixed version of os-xenapi from nova's requirements... | |
| 14:12:18 | BobBall | Made the mistake of trying to open a 1MB log file in firefox. Apparently it doesn't like a 1MB html file and has decided it needs a really long think about it | |
| 14:13:06 | johnthetubaguy | BobBall: yeah, I had to download it | |
| 14:13:13 | johnthetubaguy | BobBall: I am seeing 404 from glance | |
| 14:13:28 | BobBall | Not ideal. | |
| 14:20:04 | BobBall | Looks like glance upload is broken | |
| 14:20:50 | mriedem | BobBall: like this? | |
| 14:20:51 | mriedem | 2017-02-01 12:47:40.884 20207 ERROR nova.compute.manager [instance: 797cb75d-5ab2-4231-b972-db08257716f4] Failure: ['XENAPI_PLUGIN_FAILURE', 'upload_vhd2', 'PluginError', 'Got Permanent Error response [404] while uploading image [79f5d221-0e61-4ee9-82e4-69f507219b78] to glance [http://192.168.33.1:9292/v2/images/79f5d221-0e61-4ee9-82e4-69f507219b78/file]'] | |
| 14:21:02 | BobBall | Tis only upload, yes, not download. | |
| 14:21:12 | BobBall | I think. | |
| 14:21:12 | mriedem | what changed? | |
| 14:21:15 | BobBall | Trying to figure that out. | |
| 14:23:59 | mriedem | johnthetubaguy: sfinucan: i've flip flopped and decided to nack anymore of https://blueprints.launchpad.net/nova/+spec/use-service-tokens | |
| 14:24:01 | BobBall | first failure was 11 hours ago on change 427536 | |
| 14:24:09 | mriedem | cinder and neutron are done, let's do the glance stuff in pike | |
| 14:24:21 | johnthetubaguy | mriedem: yeah, I am fine with that, thats like Friday right? | |
| 14:24:39 | mriedem | johnthetubaguy: yeah | |
| 14:24:41 | mriedem | i think so anyway | |