| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-05 | |||
| 21:30:02 | melwitt | lol | |
| 21:30:53 | mtreinish | dansmith: https://review.openstack.org/#/c/480746/ | |
| 21:31:56 | mriedem | hey, you're welcome ^ | |
| 21:31:59 | mriedem | oops | |
| 21:32:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Blacklist test_extend_attached_volume from cells v1 job https://review.openstack.org/509907 | |
| 21:32:13 | dansmith | well, melissaml +1d it so I'm surprised it was buggy | |
| 21:32:30 | mriedem | usually pretty reliable | |
| 21:32:53 | dansmith | melwitt: jaypipes ^ | |
| 21:34:48 | openstackgerrit | Jay Pipes proposed openstack/nova master: rp: break functions out of _set_traits() https://review.openstack.org/509908 | |
| 21:35:30 | mtreinish | melwitt: hmm, on https://review.openstack.org/#/c/507976/ stestr said the blacklist didn't match anything | |
| 21:35:40 | mtreinish | you might have found a bug in it | |
| 21:36:55 | melwitt | okay :) | |
| 21:37:08 | melwitt | mtreinish: do you know wassup with this? http://logs.openstack.org/76/507976/5/check/gate-nova-python35/859471a/console.html#_2017-10-05_21_00_25_162394 | |
| 21:37:56 | mtreinish | melwitt: yeah the post-processing on results is going a bit crazy because the test runner bailed before generating any artifacts | |
| 21:38:35 | mtreinish | so all the things are trying to operate on testrepository.subunit are blowing up because that was never created | |
| 21:39:11 | mtreinish | that specific du check was there for testr because it would just pass silently if no tests were ever run | |
| 21:39:30 | melwitt | oh | |
| 21:39:42 | mtreinish | so the run tox script that zuul runs does a du to check there is actual subunit data generated | |
| 21:39:54 | melwitt | do not matching anything made it bail? | |
| 21:39:56 | mtreinish | it's not really necessary on stestr though because it fails if nothign is run | |
| 21:40:01 | melwitt | *did | |
| 21:40:17 | melwitt | I guess that didn't really make my sentence better | |
| 21:40:25 | mtreinish | melwitt: yep, it exited with an error because it didn't match anything | |
| 21:40:44 | mriedem | jaypipes: i have a sql question | |
| 21:40:50 | mtreinish | that error message needs to be fixed though, it predates other non-regex selection mechanisms | |
| 21:40:50 | melwitt | k | |
| 21:40:56 | mriedem | i want to add an index on instance_actions_events for the action_id and event_id fields, | |
| 21:41:15 | mriedem | jaypipes: when we query those, we also include the deleted column so i'm assuming we want that in the index too right? | |
| 21:42:15 | jaypipes | mriedem: you don't *have* to do that, no... especially if the deleted column has very low cardinality | |
| 21:42:30 | jaypipes | mriedem: i.e. deleted column has very few distinct values | |
| 21:42:34 | mriedem | i've just noticed that we have deleted in most of our other indexes | |
| 21:42:36 | mtreinish | melwitt: the thing I'm confused by is the selection code in stestr I basically just copy and pasted from ostestr, so I'm surprised it's behaving differently | |
| 21:42:53 | melwitt | ah, yeah. I was wondering that | |
| 21:43:02 | jaypipes | mriedem: yeah, I know we have deleted in a lot of the indexes... | |
| 21:43:14 | mriedem | jaypipes: well, the values are 0 or positive int | |
| 21:43:36 | jaypipes | mriedem: right, but most are 0. | |
| 21:43:42 | mriedem | sure | |
| 21:43:53 | mriedem | maybe i should just run it both ways and see | |
| 21:43:58 | jaypipes | mriedem: that's like have a phone book with all dan smiths in it. | |
| 21:44:02 | mriedem | currently building a devstack with 500 of these | |
| 21:44:25 | gregwork | has anyone seen where all the nova services (conductor/scheduler/api) all start barfing with: ValueError: too many values to unpack. just rolling logs of this kind of error: https://paste.fedoraproject.org/paste/Vn7-6s6INFVJa5FqWb7huQ | |
| 21:44:31 | mriedem | there are 2 matt riedemanns in the dan smith phone book | |
| 21:44:54 | gregwork | nova-manage db sync also returns that ValueError | |
| 21:45:28 | jaypipes | gregwork: you have bad sql_connection config option values. | |
| 21:45:28 | mriedem | gregwork: looks like a config issue | |
| 21:45:32 | mriedem | yup | |
| 21:45:37 | mriedem | check your [database]/connection value | |
| 21:46:30 | gregwork | in /etc/nova/nova.conf ? | |
| 21:47:16 | mriedem | yes | |
| 21:48:53 | mtreinish | melwitt: found the issue, left a review comment | |
| 21:49:02 | melwitt | looking | |
| 21:49:07 | mtreinish | I was confused there for a second, because the flag worked fine locally for me | |
| 21:49:22 | melwitt | ah, thanks | |
| 21:50:21 | openstackgerrit | melanie witt proposed openstack/nova master: Make setenv consistent for unit, func, and api-samples https://review.openstack.org/507976 | |
| 22:02:16 | gregwork | yep that was exactly it | |
| 22:02:33 | gregwork | was trying to deploy browbeat and it modified nova.conf to add +connmon to the connection= | |
| 22:02:44 | gregwork | but the install failed however the conf's had been updated | |
| 22:03:03 | gregwork | thanks! | |
| 22:21:21 | mriedem | omg here it comes | |
| 22:21:35 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add live.migration.force.complete to the legacy notification whitelist https://review.openstack.org/508877 | |
| 22:21:35 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add _wait_for_action_fail_completion to InstanceHelperMixin https://review.openstack.org/509922 | |
| 22:21:36 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add recreate test for live migrate rollback not cleaning up dest allocs https://review.openstack.org/509923 | |
| 22:21:36 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional migrate force_complete test https://review.openstack.org/509924 | |
| 22:21:37 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional for live migrate delete https://review.openstack.org/509925 | |
| 22:21:37 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Remove dest node allocations during live migration rollback https://review.openstack.org/509926 | |
| 22:48:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Note TrustedFilter deprecation in docs https://review.openstack.org/509931 | |
| 23:10:49 | mriedem | well this is weird, i stop 1 instance and it generates 3 events http://paste.openstack.org/show/622795/ | |
| 23:13:25 | mtreinish | mriedem: all the request ids are different? | |
| 23:14:12 | mriedem | and 2 of the events don't have project ids | |
| 23:14:15 | mriedem | i bet i know what this is | |
| 23:14:22 | mriedem | sync_instance_power_state periodic | |
| 23:14:45 | mriedem | the last 2 are 10 minutes apart | |
| 23:14:51 | mriedem | which is the default on that periodic | |
| 23:16:07 | mriedem | ha, yeah, because the fake driver that i'm using doesn't change the state on the fake guest when you power it off | |
| 23:16:14 | mriedem | so the db says it's stopped but the "hypervisor" doesn't | |
| 23:16:16 | mriedem | so nova stops it | |
| 23:16:24 | mriedem | sev1 fake driver bug | |
| 23:18:57 | mtreinish | mriedem: hah, nice | |
| 23:21:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Implement power_off/power_on for the FakeDriver https://review.openstack.org/509935 | |
| 23:38:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add live.migration.force.complete to the legacy notification whitelist https://review.openstack.org/508877 | |
| 23:38:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional migrate force_complete test https://review.openstack.org/509924 | |
| 23:38:23 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Add functional for live migrate delete https://review.openstack.org/509925 | |
| 23:38:23 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Remove dest node allocations during live migration rollback https://review.openstack.org/509926 | |
| #openstack-nova - 2017-10-06 | |||
| 00:09:35 | mriedem | mtreinish: does sourcing /opt/stack/new/devstack/lib/nova in grenade not also source the /opt/stack/new/devstack/local.conf? | |
| 00:11:11 | mtreinish | mriedem: probably not, in devstack the assumption is likely that is sourced well before lib/nova is called | |
| 00:12:16 | mriedem | alright | |
| 00:12:54 | mriedem | hmm, so can i even source this thing from within a post_test_hook? | |
| 00:12:54 | mriedem | http://logs.openstack.org/71/508271/1/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/659a2cd/logs/new/local.conf.txt.gz | |
| 00:13:05 | mriedem | i need to get the CELLSV2_SETUP value | |
| 00:15:19 | mriedem | seems like the only thing available to me is the ansible env vars from the d-g setup in http://logs.openstack.org/71/508271/1/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/659a2cd/logs/devstack-gate-setup-host.txt.gz | |
| 00:15:27 | mtreinish | mriedem: local.conf should be sourced by grenade already: https://github.com/openstack-dev/grenade/blob/03de9e0fc7f4fc50a00db5d547413e26cf0780dd/grenade.sh#L228 | |
| 00:15:29 | mriedem | so i can get like GRENADE_OLD_BRANCH | |
| 00:16:56 | mriedem | yeah but this is a script running in the post_test_hook | |
| 00:17:01 | mriedem | would that carry over? | |
| 00:18:31 | mriedem | http://logs.openstack.org/71/508271/1/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/659a2cd/console.html#_2017-10-04_14_36_43_557726 | |
| 00:18:35 | mtreinish | ah, ok. Yeah it won't be sourced there | |
| 00:18:45 | melwitt | mtreinish: still failing after installing python3-rados, even though it shows up in py3 pip freeze on the job. was not expecting that. and when I try locally I can import rados in py3 http://logs.openstack.org/63/509663/3/experimental/gate-tempest-dsvm-py35-full-devstack-plugin-ceph-ubuntu-xenial-nv/2361fe2/logs/screen-g-api.txt.gz?level=ERROR | |
| 00:18:51 | mriedem | ok, so i need to do https://github.com/openstack-dev/grenade/blob/03de9e0fc7f4fc50a00db5d547413e26cf0780dd/grenade.sh#L228 from within the script | |
| 00:19:23 | melwitt | back to the drawing board | |
| 00:20:34 | mtreinish | melwitt: what about rbd on py3. Looking at the glance_store code it if either rados or rbd isn't present it'll set both to None | |