Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-07-09
15:59:30 sean-k-mooney 89% is not too bad
16:00:18 sean-k-mooney mriedem: cdent care to point out what i need to chagne to get it to run everything
16:02:20 mriedem mine isn't done but this is what i have http://paste.openstack.org/show/754229/
16:02:55 mriedem i've mixed some cleanups into that
16:04:55 sean-k-mooney ok its re running now and ya the functional tests are now running too
16:05:25 sean-k-mooney actully ill add teh concurrancy stuff
16:09:27 sean-k-mooney should be done in another miniute or so just grabing a coffee
16:13:15 sean-k-mooney ya im seeing it stall with timeout near the end so we are not mocking something properly although it appears to be in the unit test which is weird
16:14:01 sean-k-mooney maybe we mess up some global state in the functional tests?
16:15:13 sean-k-mooney i am running these under py37 which also could be related but probably not
16:18:12 sean-k-mooney hum it look liek the libvirt issue might be real
16:19:16 sean-k-mooney it looks like some of our mock dont work under python 3.7 and we actully try to connect to libvirt
16:23:57 mriedem gibi: now that you're leaving or already gone for the day, i've gone through your tests https://review.opendev.org/#/c/637955/34
16:24:54 mriedem sean-k-mooney: i would not be surprised, probably missing the FakeLibvirtFixture somewhere
16:25:11 mriedem because several of the libvirt modules have a global libvirt variable which tests are supposed to stub out to be the fakelibvirt module
16:28:19 mriedem test_cleanup_volumes_pending_resize is mocking the wrong thing...
16:28:30 mriedem RBDVolumeProxy rather than RbdProxy
16:28:31 sean-k-mooney im seeing a whole bunch of placement time out where we call _get_guest_config which goes all the way down to calling _connect in nova.virt.libvirt.host
16:28:44 sean-k-mooney ya i noticed that too
16:29:13 sean-k-mooney im piping the full output to a file and ill through it up on google drive or something
16:29:39 sean-k-mooney its more or less done at this point i think just waiting for all the timeouts to fire
16:30:17 sean-k-mooney all the tox treads are more or less sleeping and then ever so foter a test times out and a bunch suceed
16:33:46 sean-k-mooney test_min_version_file_backed_bad_ram_allocation_ratio apparently modifies teh db witout without using the right fixture
16:34:53 sean-k-mooney so ya there are a whole bunc of not so nice errors showing up
16:35:03 mriedem i don't see _check_file_backed_memory_support hitting the db at all
16:35:30 sean-k-mooney my output could be interleaved
16:37:16 sean-k-mooney i have a couple of things that look like this
16:37:17 sean-k-mooney http://paste.openstack.org/show/754231/
16:38:08 mriedem that would be the libvirt host connection callback stuff that calls _set_host_enabled
16:38:31 sean-k-mooney ya which is happening because our mocking of the connection info is messed up
16:38:59 sean-k-mooney we should be useing the fake libvirt conenction there or just mocking it out entirely sicne it a unit test
16:39:06 mriedem i've had problems in the past with mocks on the libvirt Host stuff leading to weird intermittent failures
16:39:12 mriedem especially with the hasVersion type calls
16:39:35 mriedem like this:
16:39:36 sean-k-mooney ya you mentioned that on my vidoe model patchs
16:39:36 mriedem test_swap_volume_file
16:39:47 mriedem @mock.patch.object(host.Host,
16:39:47 mriedem 'has_min_version', return_value=True)
16:40:00 mriedem rather than mocking the drvr.host.has_min_version directly
16:40:30 mriedem *drvr._host.has_min_version
16:40:40 mriedem there are a billion libvirt driver unit tests so untangling that would be pretty hard
16:41:10 sean-k-mooney well we could do a find an replace
16:41:20 mriedem i don't think it's that easy
16:41:26 sean-k-mooney e.g. if there is one way we know works grep for the broken form
16:41:31 sean-k-mooney proably not
16:41:50 sean-k-mooney but if you know a way that definetly works we can fix the unstable kind as we find them
16:42:12 sean-k-mooney is there a way we could shorten these timeout for the tests by the way?
16:42:18 mriedem yes
16:42:25 sean-k-mooney e.g. globaly somehow
16:42:42 mriedem OS_TEST_TIMEOUT i think
16:42:58 mriedem it's in tox.ini
16:43:10 sean-k-mooney or time out the tests them seleves
16:43:43 sean-k-mooney that might cause issue with some of them but it would be worth trying
16:44:02 mriedem well it would make this monster coverage thing fail faster yes
16:44:13 mriedem dropping OS_TEST_TIMEOUT=60 would help
16:44:14 sean-k-mooney i was thinking more doing set.flags in the base test case
16:44:29 mriedem nova.conf doesn't have anything to do with test execution timeout...
16:44:47 sean-k-mooney i was wondering if we could modify libvirt timeouts in nova.conf
16:45:02 mriedem nope
16:45:11 openstackgerrit Merged openstack/nova master: Implement update_provider_tree https://review.opendev.org/667417
16:45:20 mriedem configuring libvirt is kind of a libvirt.conf thing right?
16:45:50 sean-k-mooney ya i guess i think these timeout are comeing form teh python binding or the c lib rather then libvirt
16:47:03 sean-k-mooney ill give it another 15mins and if its still timing out ten ill run with OS_TEST_TIMEOUT=30 currently its timing out after 160s
16:50:40 sean-k-mooney actully i might make that OS_TEST_TIMEOUT=8 im not seeing any test over about 4.5 seconds
16:51:09 sean-k-mooney oh there is 1 at 13s
16:55:52 dansmith artom: I assume you're working on feedback on the resize revert patches yeah?
17:09:49 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: coverage job changes https://review.opendev.org/669912
17:12:50 openstackgerrit Merged openstack/nova master: xenapi: implement update_provider_tree https://review.opendev.org/667435
17:12:58 openstackgerrit Merged openstack/nova master: Deprecate non-update_provider_tree compat code https://review.opendev.org/667442
17:13:06 Sundar gibi, efried: Re. https://review.opendev.org/#/c/657464/, there was some discussion over naming, which got resolved in the review of the spec https://review.opendev.org/#/c/603955/12/specs/train/approved/nova-cyborg-interaction.rst@307 . Can we resume the review of 657464?
17:13:07 openstackgerrit Merged openstack/nova master: Update COMPUTE_STATUS_DISABLED from set_host_enabled compute call https://review.opendev.org/668743
17:14:02 efried Sundar: As far as I'm concerned, absolutely. Not much I can do though, since I own the patch :)
17:31:49 openstackgerrit Merged openstack/nova master: Add placement request pre-filter compute_status_filter https://review.opendev.org/668752
17:31:56 openstackgerrit Merged openstack/nova master: Refactor HostAPI.service_update https://review.opendev.org/668756
17:32:05 openstackgerrit Merged openstack/nova master: Sync COMPUTE_STATUS_DISABLED from API https://review.opendev.org/654596
17:38:33 artom dansmith, yeah
17:41:43 dansmith cool
17:54:07 openstackgerrit Merged openstack/nova master: Follow up for pre-filter-disabled-computes series https://review.opendev.org/668986
18:51:04 mriedem +2 on the host/hypervisor_hostname on server create api change again https://review.opendev.org/#/c/645520/
19:13:49 openstackgerrit Merged openstack/nova master: Cleanup quota user docs https://review.opendev.org/662573
19:16:19 openstackgerrit Matt Riedemann proposed openstack/nova master: nova-lvm: Disable [validation]/run_validation in tempest.conf https://review.opendev.org/662176
19:30:02 openstackgerrit Matt Riedemann proposed openstack/nova master: Update AZ admin doc to mention the new way to specify hosts https://review.opendev.org/666767
19:34:37 cdent mriedem: I think you already know this, but my coverage run locked up and stayed locked (until I forced it dead just now) in nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_check_can_live_migrate_dest_file_backed_discard_bad_libvirt
19:37:43 mriedem i also killed mine
19:37:49 mriedem but posted a change
19:37:56 mriedem which will probably timeout :)
19:39:51 mriedem ah yeah
19:39:51 mriedem @mock.patch.object(fakelibvirt.Connection, 'getVersion')
19:39:51 mriedem @mock.patch.object(fakelibvirt.Connection, 'getLibVersion')
19:40:01 mriedem but that doesn't mock out conn = self.get_connection()
19:50:35 sean-k-mooney ya i have seen the same
19:51:57 sean-k-mooney i ws just checking it before i pip out to the shops to get dinner but i was thinking maybe we could run the test in two batchs
19:52:33 sean-k-mooney run the funtional test only first then unit tests only then have coverage combine the combine the two into one set of results
19:54:41 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: remove unused Service.get_by_compute_host mocks https://review.opendev.org/669934
20:01:06 mriedem sean-k-mooney: huh https://github.com/nedbat/coveragepy/blob/6b72dcbd98c69a7e8c5333d2926c7f871161db7e/coverage/collector.py#L115
20:01:09 mriedem the docs are really not clear about that
20:04:20 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: coverage job changes https://review.opendev.org/669912
20:06:28 mriedem efried: can you re-approve https://review.opendev.org/#/c/666042/ since i just had to rebase and fix the pep8 thing due to the newly enforced hacking rules
20:06:58 efried ...
20:08:22 efried done

Earlier   Later