| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-28 | |||
| 00:56:15 | mriedem | yeah - related to that https://review.openstack.org/#/c/424574/ | |
| 00:56:20 | mriedem | if you want to +W if you're happy with it | |
| 00:56:28 | mriedem | i'm not sure about time.clock() vs time.time() in there | |
| 00:56:36 | mriedem | i feel like pythonistas would flame that up | |
| 00:57:24 | melwitt | I was looking at that earlier and wondering, is there really no way to do notifications synchronously for tests? it's using eventlet etc | |
| 00:58:33 | superdan | maybe later, I'm breaking for at bit | |
| 00:59:03 | mriedem | melwitt: do we spawn(_n) the notifications> | |
| 00:59:04 | mriedem | ? | |
| 00:59:33 | mriedem | we could set those tests to REQUIRES_LOCKING=True | |
| 00:59:43 | mriedem | if they use spawn/spawn_n we can make those synchronous too | |
| 01:00:21 | melwitt | mriedem: I don't really understand it yet, but was looking at nova/tests/unit/fake_notifier.py | |
| 01:00:39 | mriedem | we could also use SpawnIsSynchronousFixture | |
| 01:01:01 | melwitt | I think you already added that | |
| 01:02:09 | melwitt | in the FakeNotifier, the SUBSCRIBERS are threading.Events that it waits on | |
| 01:03:09 | melwitt | well, if you call wait_for_versioned_notification(), which is what the proposed patch is doing, I think | |
| 01:13:10 | melwitt | we have a RPCFixture which configures the driver to the 'fake' one provided by oslo.messaging, but I don't see that we do anything similar for notifications. there's a 'test' driver for testing and I wonder if we could use that somehow https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/notify/_impl_test.py | |
| 01:14:44 | melwitt | I don't know what driver is currently being used. there's no default in oslo.messaging and I don't find it configured in nova anywhere | |
| 01:16:13 | mriedem | idk either, gibi probably does | |
| 01:17:24 | mriedem | single-node grenade is happy now on the scheduler change | |
| 01:18:30 | melwitt | yeah, I should ask on the review. just seems weird to have to wait for an async notification in a test | |
| 01:18:54 | mriedem | and grenade multinode | |
| 01:22:05 | ptcBRO | Hey guys, I have instances with invalid data in the instance.info_cache. Is there a recommended way to invalidate this cache? | |
| 01:38:55 | mriedem | ptcBRO: there is a periodic task on the compute node that will refresh the cache automatically | |
| 01:39:39 | mriedem | otherwise i think you have to reset it in the db directly, i don't know of a REST API to refresh it manually | |
| 01:40:07 | ptcBRO | mriedem: does that periodic task exist in Juno? | |
| 01:40:20 | mriedem | ptcBRO: yeah i think so | |
| 01:40:31 | mriedem | heal_info_cache or something like that | |
| 01:40:34 | mriedem | heal_instance_info_cache | |
| 01:41:11 | mriedem | ptcBRO: https://github.com/openstack/nova/blob/juno-eol/nova/compute/manager.py#L5368 | |
| 01:41:25 | ptcBRO | mriedem: thank you | |
| 01:43:09 | mriedem | melwitt: i'd love it if we could get this in https://review.openstack.org/#/c/403925/ - i get tired of the InstanceNotFound stacktraces all over the logs in CI runs | |
| 01:48:49 | melwitt | mriedem: I've seen those and would also like to get rid of them. will review | |
| 02:35:32 | diana_clarke | man, did I ever mess up this rebase... poor friday night choices | |
| 02:57:15 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 delete_cell https://review.openstack.org/421459 | |
| 02:57:15 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 verify_instance https://review.openstack.org/421436 | |
| 02:57:16 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 list_cells https://review.openstack.org/421458 | |
| 02:57:16 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 discover_hosts https://review.openstack.org/421442 | |
| 02:57:17 | openstackgerrit | Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 create_cell https://review.openstack.org/421444 | |
| 03:01:47 | oomichi | mriedem: I saw your mail: http://lists.openstack.org/pipermail/openstack-dev/2017-January/111137.html | |
| 03:02:11 | oomichi | mriedem: I could not find any related changes since 1/21 | |
| 03:03:35 | oomichi | devstack changes live_migration False on tempest.conf, but devsack-gate makes it True. And there are not any changes related to that recently | |
| 03:12:00 | mriedem | oomichi: yeah it's a mystery | |
| 03:12:12 | mriedem | i checked the git logs on devstack, grenade, nova, tempest and d-g | |
| 03:16:47 | oomichi | mriedem: I guess https://review.openstack.org/426461 can help | |
| 03:17:15 | oomichi | mriedem: but I still am not sure why this happens recently | |
| 03:17:36 | mriedem | superdan: so the nova changes are approved, what should we do about the grenade change? | |
| 03:17:51 | mriedem | i guess wait for sdague on monday? | |
| 03:20:25 | oomichi | mriedem: oh, mine is wrong | |
| 03:21:17 | mriedem | oomichi: d-g is setting it correctly http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/console.html#_2017-01-27_23_16_09_271784 | |
| 03:21:30 | mriedem | it's just that for whatever reason, it's not getting set in tempest.conf from within devstack | |
| 03:22:53 | oomichi | nice hint. Yeah that seems mystery.. | |
| 03:29:07 | mriedem | i'm wondering if the devstack-tools thing is messing something up | |
| 03:29:13 | mriedem | but i don't know enough about that change | |
| 03:29:35 | mriedem | but for example, NOVA_ALLOW_MOVE_TO_SAME_HOST=False is in local.conf | |
| 03:29:40 | mriedem | http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/logs/new/local.conf.txt.gz | |
| 03:29:52 | mriedem | https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L541 | |
| 03:29:52 | mriedem | but the other isn't for live migration | |
| 03:30:03 | mriedem | and the difference is one uses export and the other doesn't | |
| 03:39:24 | oomichi | the other multinode tempest jobs set live_migration True correctly like http://logs.openstack.org/16/411816/5/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/4a95eca/logs/tempest_conf.txt.gz | |
| 14:12:10 | openstackgerrit | ChangBo Guo(gcb) proposed openstack/nova: Skip unit tests for SSL + py3 https://review.openstack.org/426505 | |
| 14:12:10 | openstackgerrit | ChangBo Guo(gcb) proposed openstack/nova: Enable more virt.xenapi tests on Python 3 https://review.openstack.org/414421 | |
| 17:37:13 | mriedem | mikal: i see the vendordata v2 patches all merged and have release notes, which is cool. we should update the devref at some point (not an immediate issue) for the new features http://docs.openstack.org/developer/nova/vendordata.html | |
| 17:51:01 | mriedem | johnthetubaguy: i guess we'll just defer the rest of this to pike https://review.openstack.org/#/q/topic:bp/prep-for-network-aware-scheduling-ocata+status:open | |
| #openstack-nova - 2017-01-29 | |||
| 00:35:44 | openstackgerrit | David Medberry proposed openstack/nova: Explain how allow_resize_to_same_host is useful https://review.openstack.org/426531 | |
| 00:39:20 | openstackgerrit | Emilien Macchi proposed openstack/nova: Revert "Move instance creation to conductor" https://review.openstack.org/426532 | |
| 08:33:35 | jschwarz | hi | |
| 14:28:50 | openstackgerrit | Chris Dent proposed openstack/nova: Add more debug logging on RP inventory update failures https://review.openstack.org/414230 | |
| 14:28:51 | openstackgerrit | Chris Dent proposed openstack/nova: Add more debug logging on RP inventory delete failures https://review.openstack.org/426290 | |
| 15:00:03 | openstackgerrit | Emilien Macchi proposed openstack/nova: Revert "Move instance creation to conductor" https://review.openstack.org/426532 | |
| 15:00:44 | EmilienM | superdan: I confirm https://review.openstack.org/#/c/319379/ breaks TripleO CI | |
| 15:01:10 | EmilienM | superdan: after this patch, we weren't able to spawn servers | |
| 15:10:17 | openstackgerrit | Ildiko Vancsa proposed openstack/nova: Implement new attach/detach Cinder flow https://review.openstack.org/330285 | |
| 15:10:19 | openstackgerrit | Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358 | |
| 15:11:21 | openstackgerrit | Ildiko Vancsa proposed openstack/nova: Implement new attach/detach Cinder flow https://review.openstack.org/330285 | |
| 15:11:23 | openstackgerrit | Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358 | |
| 17:04:40 | superdan | EmilienM: because you don't have cells records in place or something? | |
| 17:04:52 | EmilienM | superdan: we should have | |
| 17:05:06 | openstack | Launchpad bug 1660160 in tripleo "No host-to-cell mapping found for selected host" [Critical,Triaged] | |
| 17:05:06 | EmilienM | dansmith: I filed a bug https://bugs.launchpad.net/tripleo/+bug/1660160 | |
| 17:05:28 | dansmith | EmilienM: that message means you don't | |
| 17:06:47 | EmilienM | we have nova-cell_v2-cell-default | |
| 17:06:57 | EmilienM | http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/undercloud_install.txt.gz#_2017-01-27_18_24_40_000 | |
| 17:07:02 | EmilienM | let me check what it executes | |
| 17:07:31 | EmilienM | http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/undercloud_install.txt.gz#_2017-01-27_18_24_40_000 | |
| 17:07:37 | EmilienM | damn, wrong link | |
| 17:07:40 | EmilienM | https://github.com/openstack/puppet-nova/blob/master/manifests/cell_v2/cell.pp#L45 | |
| 17:07:44 | EmilienM | this is what we executed | |
| 17:08:00 | EmilienM | nova-manage ${extra_params} cell_v2 create_cell --name=default ${transport_url_real} ${database_connection_real} | |
| 17:08:35 | dansmith | EmilienM: the error message you linked in the bug about no compute node record is resource tracker, nothing to do with cellsv2 | |
| 17:10:17 | EmilienM | dansmith: you know better than me, to be honest I don't know. I just know that something broke again for us with a very recent commit | |
| 17:10:22 | EmilienM | and you can see it here: http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-01-27_19_01_00_621 | |
| 17:12:00 | dansmith | EmilienM: that message comes from resource tracker on compute | |
| 17:12:44 | EmilienM | dansmith: I tried to revert https://review.openstack.org/#/c/319379/ and run experimental pipeline, and tripleo worked again. | |
| 17:13:00 | EmilienM | so we'll just pin nova in RDO for now. | |
| 17:13:50 | dansmith | EmilienM: it passed multinode devstack and grenade, so it must be something in the tripleo job config that isn't set right.. why do you think just reverting the patch is the way to go? | |
| 17:14:30 | EmilienM | dansmith: it is not the way to go | |
| 17:15:17 | EmilienM | and we'll unpin when someone will help to figure what's wrong, we have zero clue at this time. We are not experts enough in Nova | |
| 17:20:05 | EmilienM | enjoy! | |
| 18:25:34 | mriedem | does anyone else enjoy doing taxes as much as i do?! | |
| 18:35:41 | tbachman | mriedem: clearly not, given how early you’ve started them ;) | |