Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-01-28
01:00:39 mriedem we could also use SpawnIsSynchronousFixture
01:01:01 melwitt I think you already added that
01:02:09 melwitt in the FakeNotifier, the SUBSCRIBERS are threading.Events that it waits on
01:03:09 melwitt well, if you call wait_for_versioned_notification(), which is what the proposed patch is doing, I think
01:13:10 melwitt we have a RPCFixture which configures the driver to the 'fake' one provided by oslo.messaging, but I don't see that we do anything similar for notifications. there's a 'test' driver for testing and I wonder if we could use that somehow https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/notify/_impl_test.py
01:14:44 melwitt I don't know what driver is currently being used. there's no default in oslo.messaging and I don't find it configured in nova anywhere
01:16:13 mriedem idk either, gibi probably does
01:17:24 mriedem single-node grenade is happy now on the scheduler change
01:18:30 melwitt yeah, I should ask on the review. just seems weird to have to wait for an async notification in a test
01:18:54 mriedem and grenade multinode
01:22:05 ptcBRO Hey guys, I have instances with invalid data in the instance.info_cache. Is there a recommended way to invalidate this cache?
01:38:55 mriedem ptcBRO: there is a periodic task on the compute node that will refresh the cache automatically
01:39:39 mriedem otherwise i think you have to reset it in the db directly, i don't know of a REST API to refresh it manually
01:40:07 ptcBRO mriedem: does that periodic task exist in Juno?
01:40:20 mriedem ptcBRO: yeah i think so
01:40:31 mriedem heal_info_cache or something like that
01:40:34 mriedem heal_instance_info_cache
01:41:11 mriedem ptcBRO: https://github.com/openstack/nova/blob/juno-eol/nova/compute/manager.py#L5368
01:41:25 ptcBRO mriedem: thank you
01:43:09 mriedem melwitt: i'd love it if we could get this in https://review.openstack.org/#/c/403925/ - i get tired of the InstanceNotFound stacktraces all over the logs in CI runs
01:48:49 melwitt mriedem: I've seen those and would also like to get rid of them. will review
02:35:32 diana_clarke man, did I ever mess up this rebase... poor friday night choices
02:57:15 openstackgerrit Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 verify_instance https://review.openstack.org/421436
02:57:15 openstackgerrit Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 delete_cell https://review.openstack.org/421459
02:57:16 openstackgerrit Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 discover_hosts https://review.openstack.org/421442
02:57:16 openstackgerrit Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 list_cells https://review.openstack.org/421458
02:57:17 openstackgerrit Diana Clarke proposed openstack/nova: nova-manage docs: cell_v2 create_cell https://review.openstack.org/421444
03:01:47 oomichi mriedem: I saw your mail: http://lists.openstack.org/pipermail/openstack-dev/2017-January/111137.html
03:02:11 oomichi mriedem: I could not find any related changes since 1/21
03:03:35 oomichi devstack changes live_migration False on tempest.conf, but devsack-gate makes it True. And there are not any changes related to that recently
03:12:00 mriedem oomichi: yeah it's a mystery
03:12:12 mriedem i checked the git logs on devstack, grenade, nova, tempest and d-g
03:16:47 oomichi mriedem: I guess https://review.openstack.org/426461 can help
03:17:15 oomichi mriedem: but I still am not sure why this happens recently
03:17:36 mriedem superdan: so the nova changes are approved, what should we do about the grenade change?
03:17:51 mriedem i guess wait for sdague on monday?
03:20:25 oomichi mriedem: oh, mine is wrong
03:21:17 mriedem oomichi: d-g is setting it correctly http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/console.html#_2017-01-27_23_16_09_271784
03:21:30 mriedem it's just that for whatever reason, it's not getting set in tempest.conf from within devstack
03:22:53 oomichi nice hint. Yeah that seems mystery..
03:29:07 mriedem i'm wondering if the devstack-tools thing is messing something up
03:29:13 mriedem but i don't know enough about that change
03:29:35 mriedem but for example, NOVA_ALLOW_MOVE_TO_SAME_HOST=False is in local.conf
03:29:40 mriedem http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/logs/new/local.conf.txt.gz
03:29:52 mriedem but the other isn't for live migration
03:29:52 mriedem https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L541
03:30:03 mriedem and the difference is one uses export and the other doesn't
03:39:24 oomichi the other multinode tempest jobs set live_migration True correctly like http://logs.openstack.org/16/411816/5/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/4a95eca/logs/tempest_conf.txt.gz
14:12:10 openstackgerrit ChangBo Guo(gcb) proposed openstack/nova: Enable more virt.xenapi tests on Python 3 https://review.openstack.org/414421
14:12:10 openstackgerrit ChangBo Guo(gcb) proposed openstack/nova: Skip unit tests for SSL + py3 https://review.openstack.org/426505
17:37:13 mriedem mikal: i see the vendordata v2 patches all merged and have release notes, which is cool. we should update the devref at some point (not an immediate issue) for the new features http://docs.openstack.org/developer/nova/vendordata.html
17:51:01 mriedem johnthetubaguy: i guess we'll just defer the rest of this to pike https://review.openstack.org/#/q/topic:bp/prep-for-network-aware-scheduling-ocata+status:open
#openstack-nova - 2017-01-29
00:35:44 openstackgerrit David Medberry proposed openstack/nova: Explain how allow_resize_to_same_host is useful https://review.openstack.org/426531
00:39:20 openstackgerrit Emilien Macchi proposed openstack/nova: Revert "Move instance creation to conductor" https://review.openstack.org/426532
08:33:35 jschwarz hi
14:28:50 openstackgerrit Chris Dent proposed openstack/nova: Add more debug logging on RP inventory update failures https://review.openstack.org/414230
14:28:51 openstackgerrit Chris Dent proposed openstack/nova: Add more debug logging on RP inventory delete failures https://review.openstack.org/426290
15:00:03 openstackgerrit Emilien Macchi proposed openstack/nova: Revert "Move instance creation to conductor" https://review.openstack.org/426532
15:00:44 EmilienM superdan: I confirm https://review.openstack.org/#/c/319379/ breaks TripleO CI
15:01:10 EmilienM superdan: after this patch, we weren't able to spawn servers
15:10:17 openstackgerrit Ildiko Vancsa proposed openstack/nova: Implement new attach/detach Cinder flow https://review.openstack.org/330285
15:10:19 openstackgerrit Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358
15:11:21 openstackgerrit Ildiko Vancsa proposed openstack/nova: Implement new attach/detach Cinder flow https://review.openstack.org/330285
15:11:23 openstackgerrit Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358
17:04:40 superdan EmilienM: because you don't have cells records in place or something?
17:04:52 EmilienM superdan: we should have
17:05:06 EmilienM dansmith: I filed a bug https://bugs.launchpad.net/tripleo/+bug/1660160
17:05:06 openstack Launchpad bug 1660160 in tripleo "No host-to-cell mapping found for selected host" [Critical,Triaged]
17:05:28 dansmith EmilienM: that message means you don't
17:06:47 EmilienM we have nova-cell_v2-cell-default
17:06:57 EmilienM http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/undercloud_install.txt.gz#_2017-01-27_18_24_40_000
17:07:02 EmilienM let me check what it executes
17:07:31 EmilienM http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/undercloud_install.txt.gz#_2017-01-27_18_24_40_000
17:07:37 EmilienM damn, wrong link
17:07:40 EmilienM https://github.com/openstack/puppet-nova/blob/master/manifests/cell_v2/cell.pp#L45
17:07:44 EmilienM this is what we executed
17:08:00 EmilienM nova-manage ${extra_params} cell_v2 create_cell --name=default ${transport_url_real} ${database_connection_real}
17:08:35 dansmith EmilienM: the error message you linked in the bug about no compute node record is resource tracker, nothing to do with cellsv2
17:10:17 EmilienM dansmith: you know better than me, to be honest I don't know. I just know that something broke again for us with a very recent commit
17:10:22 EmilienM and you can see it here: http://logs.openstack.org/31/426231/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/915aeba/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-01-27_19_01_00_621
17:12:00 dansmith EmilienM: that message comes from resource tracker on compute
17:12:44 EmilienM dansmith: I tried to revert https://review.openstack.org/#/c/319379/ and run experimental pipeline, and tripleo worked again.
17:13:00 EmilienM so we'll just pin nova in RDO for now.
17:13:50 dansmith EmilienM: it passed multinode devstack and grenade, so it must be something in the tripleo job config that isn't set right.. why do you think just reverting the patch is the way to go?
17:14:30 EmilienM dansmith: it is not the way to go
17:15:17 EmilienM and we'll unpin when someone will help to figure what's wrong, we have zero clue at this time. We are not experts enough in Nova
17:20:05 EmilienM enjoy!
18:25:34 mriedem does anyone else enjoy doing taxes as much as i do?!
18:35:41 tbachman mriedem: clearly not, given how early you’ve started them ;)
18:54:10 amrith andreykurilin, ping. please take a look at http://logs.openstack.org/35/426535/3/check/gate-trove-scenario-dsvm-mysql-multi-ubuntu-xenial/16eaf0c/console.html#_2017-01-29_17_52_32_209313. something appears to be getting mixed up between novaclient and keystone and I'm not sure what that could be. this particular failure seems to be out of trove's direct control (I think). but something trove is doing may be the thing
18:54:10 amrith that is exacerbating this.
20:26:26 andreykurilin amrith: will contact you at trove channel
#openstack-nova - 2017-01-30
02:34:11 openstackgerrit Takashi NATSUME proposed openstack/nova-specs: Enable cold migration with target host https://review.openstack.org/426636
02:38:57 openstackgerrit Takashi NATSUME proposed openstack/nova: Enable cold migration with target host(1/2) https://review.openstack.org/408955
02:39:25 openstackgerrit Takashi NATSUME proposed openstack/nova: Enable cold migration with target host(1/2) https://review.openstack.org/408955
02:40:05 openstackgerrit Takashi NATSUME proposed openstack/nova: Enable cold migration with target host(2/2) https://review.openstack.org/408964
02:40:22 openstackgerrit Takashi NATSUME proposed openstack/nova: Enable cold migration with target host(2/2) https://review.openstack.org/408964
02:41:57 openstackgerrit Takashi NATSUME proposed openstack/nova-specs: Enable cold migration with target host https://review.openstack.org/426636
02:44:02 openstackgerrit Takashi NATSUME proposed openstack/nova: api-ref: Add parameters in cold migrate action https://review.openstack.org/410042
02:44:08 openstackgerrit Takashi NATSUME proposed openstack/nova: api-ref: Add parameters in cold migrate action https://review.openstack.org/410042

Earlier   Later