| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-17 | |||
| 18:36:26 | mriedem | a shiny nickel to whoever can figure out why my unit test is failing in ^ | |
| 18:37:36 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add policy-docs spec https://review.openstack.org/433010 | |
| 18:38:10 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add policy-remove-scope-checks spec https://review.openstack.org/433037 | |
| 18:38:16 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add additional-default-policy-roles spec https://review.openstack.org/427872 | |
| 18:39:47 | sdague | mriedem: because something else managed to set that global already | |
| 18:39:57 | mriedem | gah | |
| 18:40:10 | mriedem | ok i knew i needed to mock that out at some point in here too | |
| 18:40:23 | mriedem | sdague: i owe you a nickel | |
| 18:40:25 | mriedem | :) | |
| 18:50:58 | openstackgerrit | Pushkar Umaranikar proposed openstack/nova master: Placement api: set custom json_error_formatter in resource_class https://review.openstack.org/430463 | |
| 18:52:05 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP(5): PowerVM driver: ovs vif https://review.openstack.org/422512 | |
| 18:57:07 | zzfoo | Any ideas why I'm getting this trace on live migrate: http://paste.openstack.org/show/599457/ | |
| 18:57:08 | zzfoo | MigrationError_Remote: Migration error: failed to detect a valid IP address from '' | |
| 18:57:48 | thorst_ | zzfoo: not sure...that looks like a controller (possibly AMQP) error? | |
| 19:00:25 | artom | zzfoo, it almost looks as though one of your computes disappeared from the network | |
| 19:01:05 | zzfoo | The exception is actually even longer here's the full thing: http://paste.openstack.org/show/599459/ | |
| 19:03:51 | cburgess | mriedem *ping* | |
| 19:03:54 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP(6): Introduce base PowerVM driver (ssp drv) https://review.openstack.org/422594 | |
| 19:04:19 | mriedem | cburgess: pong | |
| 19:04:51 | zzfoo | on the other side of the live migrate it gets this error: http://paste.openstack.org/show/599460/ | |
| 19:05:06 | cburgess | mriedem Food stuff: So neither of those BBQ places take reservations, we could try and get in. I would recommend trying Sweet Auburn as its a tad closer. But.. both at 3 miles away and thats going to be a long walk, also as edleafe pointed out there are some non-meat eaters. | |
| 19:05:24 | mriedem | sdague: yeah i suppose we don't need to even setup the ksa session stuff for vendordata if CONF.api.vendordata_dynamic_targets is not set | |
| 19:05:28 | mriedem | since we won't have any targets to hit | |
| 19:05:29 | cburgess | mriedem So... another option would be https://www.tedsmontanagrill.com Its less then a mile and has a more varied menu. | |
| 19:06:03 | superdan | mriedem: I think you might have looked at this "cell not found for instance" error before: http://logs.openstack.org/62/433362/5/check/gate-tempest-dsvm-cells-ubuntu-xenial/ac9584a/logs/screen-n-cell-child.txt.gz#_2017-02-16_22_40_43_882 | |
| 19:06:46 | superdan | mriedem: from what I see there the instance is getting deleted in the middle of something, so maybe that is expected, but I'm seeing it bubble out of the api nine times there | |
| 19:07:05 | superdan | mriedem: so it must be related to my patch, but I'm not sure why | |
| 19:07:32 | mriedem | cburgess: sure, that's a 10 min walk from the sheraton, decent review | |
| 19:07:34 | mriedem | *reviews | |
| 19:07:56 | mriedem | superdan: hmm cells v1 yeah | |
| 19:08:06 | mriedem | superdan: it is a negative test... | |
| 19:08:13 | cburgess | mriedem Yeah there are a few in ATL I've been to one twice and its not bad. The BBQ is better but Ted's is closer and I can try and see if we can get a reso. I'll call them today. | |
| 19:08:24 | superdan | mriedem: yeah, | |
| 19:08:40 | openstackgerrit | Sean Dague proposed openstack/nova master: Verify project_id when quotas are checked https://review.openstack.org/435010 | |
| 19:08:40 | openstackgerrit | Sean Dague proposed openstack/nova master: Verify project id for flavor access calls https://review.openstack.org/435432 | |
| 19:08:48 | mriedem | cburgess: it doesn't look like ted's is vegan/vegetarian/gluten free friendly either | |
| 19:08:59 | mriedem | but maybe they have an eraser people can eat | |
| 19:09:11 | superdan | mriedem: so maybe that trace is incidental to the negative test and these others are silently failing I guess | |
| 19:09:43 | mriedem | yeah it looks like http://logs.openstack.org/62/433362/5/check/gate-tempest-dsvm-cells-ubuntu-xenial/ac9584a/console.html#_2017-02-16_22_58_39_396399 is the test failure | |
| 19:09:46 | mriedem | the snapshot test | |
| 19:09:51 | mriedem | i've seen that one fail plenty in the cells v1 job | |
| 19:10:17 | mriedem | Details: {u'code': 404, u'message': u'Cell is not known for instance 518086f2-4bf8-412e-9758-8d66947b8668'} | |
| 19:10:17 | superdan | yeah | |
| 19:10:17 | mriedem | although i'm seeing lots of this in the console log | |
| 19:10:23 | superdan | right | |
| 19:10:25 | mriedem | i guess that's what you're debugging | |
| 19:10:28 | cburgess | mriedem THey have a gluten free menu https://www.tedsmontanagrill.com/menu/download/TMGGlutenFreeMenu.pdf | |
| 19:10:28 | superdan | that's a cellsv1 error | |
| 19:10:29 | superdan | yeah | |
| 19:11:10 | cburgess | mriedem They have a lot of fish and salads. Vegans though will probably have a hard time but I'm not sure of a vegan place in downtown (the BBQ places surely wouldn't work either). | |
| 19:11:45 | mriedem | there may be some random roots growing between buildings downtown | |
| 19:12:07 | cburgess | mriedem Indeed | |
| 19:12:10 | superdan | mriedem: that patch is only changing the rest api which really shouldn't know or care or influence any of the cells stuff below | |
| 19:12:21 | superdan | so maybe I'll recheck it and see if it's the same set of failures or not | |
| 19:32:05 | openstackgerrit | John L. Villalovos proposed openstack/nova master: Use flake8-import-order https://review.openstack.org/435575 | |
| 19:37:12 | openstackgerrit | Sean Dague proposed openstack/nova master: Verify project_id when quotas are checked https://review.openstack.org/435010 | |
| 19:37:12 | openstackgerrit | Sean Dague proposed openstack/nova master: Verify project id for flavor access calls https://review.openstack.org/435432 | |
| 19:55:11 | melwitt | mriedem, superdan: I have a patch up for a blocker migration that operators can get stuck on upgrading from mitaka -> newton if they have soft-deleted records https://review.openstack.org/435546 | |
| 19:55:44 | superdan | oh yeah | |
| 19:55:56 | superdan | mriedem: that one needs to get in | |
| 20:00:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Only create vendordata_dynamic ksa session if needed https://review.openstack.org/435563 | |
| 20:00:20 | mriedem | i don't want to actually think today | |
| 20:00:34 | mriedem | that needs to go back to stable/newton? | |
| 20:00:44 | mriedem | because we haven't released https://review.openstack.org/#/c/434928/ yet | |
| 20:01:24 | superdan | it does yeah | |
| 20:01:33 | mriedem | btw, this just again goes to show how much no one is event close to trunk | |
| 20:01:49 | superdan | mriedem: well, it's actually probably not very common | |
| 20:01:59 | superdan | mriedem: but definitely hit-able | |
| 20:02:11 | mriedem | let me read up on the bug report and such | |
| 20:02:33 | superdan | it would only happen if you completely decommissioned compute nodes and never replaced them | |
| 20:03:23 | melwitt | it can also happen if you've ever deleted aggregates or pci devices. it's just that wasn't the case in our real world example | |
| 20:04:09 | superdan | melwitt: yeah, true I guess I forgot about those since the CN was the issue in the bug | |
| 20:05:04 | melwitt | superdan: yeah. I was debating whether to fix it only for compute_nodes or for the other tables too. I figured it's the same problem, and that assumes no apis ever read those soft-deleted records for anything later. I don't think they do | |
| 20:05:20 | mriedem | in the red hat bugzilla i guess | |
| 20:05:31 | mriedem | so i guess, | |
| 20:05:38 | superdan | was the aggregate move to api in ocata? | |
| 20:06:00 | melwitt | I thought it was earlier | |
| 20:06:08 | melwitt | I mean, it's definitely before ocata | |
| 20:06:09 | superdan | melwitt: if so we can't have deleted aggregates | |
| 20:06:11 | mriedem | newton | |
| 20:06:23 | mriedem | https://github.com/openstack/nova/commit/35748e7a9b3445a60522e088e38bf25e6bfef25e | |
| 20:06:30 | mriedem | i was thinking it's odd to update the migration itself, | |
| 20:06:34 | mriedem | but i guess if you're stuck on 330, | |
| 20:06:40 | mriedem | then you can't get past that anyway | |
| 20:06:41 | superdan | I guess that move happens after the sync though, if they both happened in newton | |
| 20:06:52 | superdan | mriedem: yeah | |
| 20:07:05 | superdan | mriedem: the other option is to convert all the deleted records, but that's kinda silly | |
| 20:07:18 | melwitt | there's a REST API for deleting an aggregate https://developer.openstack.org/api-ref/compute/?expanded=delete-aggregate-detail#delete-aggregate | |
| 20:07:22 | superdan | especially if we're about to delete them in the case of aggregates | |
| 20:07:31 | melwitt | so if that connects to an Aggregate.destroy(), that would do it | |
| 20:07:34 | superdan | melwitt: right, but once they're in the api db they can;t be soft deleted | |
| 20:07:43 | melwitt | ah, that's what you meant. yeah | |
| 20:07:55 | mriedem | so in mitaka i deleted a compute node, | |
| 20:08:14 | mriedem | actually i need to see the online migration code first | |
| 20:08:25 | melwitt | yeah, I have links up still I think | |
| 20:08:49 | melwitt | the online migration only migrates non-soft-deleted things and the blocker doesn't filter on deleted=0 so it just gets everything | |
| 20:09:00 | mriedem | what does https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L595 have to do with compute nodes? | |
| 20:09:36 | mriedem | you guys got some out of tree online migrations? | |
| 20:09:58 | mriedem | or is it just this? https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L597 | |
| 20:10:03 | mriedem | db.aggregate_uuids_online_data_migration, | |