Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-12-06
16:26:10 openstackgerrit Matt Riedemann proposed openstack/nova master: Make build_request_spec return an actual RequestSpec https://review.opendev.org/697686
16:27:31 gibi efried_afk: I think stack trace in the log is missleading. At the end there is a dropped exception that seems relevant
16:27:34 gibi Original exception being dropped: ['Traceback (most recent call last):\n', ' File "nova/conductor/manager.py", line 1532, in schedule_and_build_instances\n scheduler_utils.fill_provider_mapping(request_spec, host)\n', ' File "nova/scheduler/utils.py", line 1234, in fill_provider_mapping\n mappings = jsonutils.loads(host_selection.allocation_request)[\'mappings\']\n', "KeyError: 'mappings'\n"]
16:32:03 gibi efried_afk: replied in the review. There is some a_c responses used by that test which does not have mapping key
16:37:34 openstackgerrit Matt Riedemann proposed openstack/nova master: Make build_request_spec return an actual RequestSpec https://review.opendev.org/697686
16:42:19 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181
16:42:20 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Unify '_build_minimal_create_server_request' implementations https://review.opendev.org/695024
16:42:20 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Remove 'get_invalid_image' https://review.opendev.org/697454
16:42:21 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Make '_IntegratedTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182
16:42:21 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add unified '_build_server' helper function https://review.opendev.org/697537
16:42:22 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add unified '_(build|create)_flavor' helper functions https://review.opendev.org/697538
16:42:22 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add '_delete_server' to 'InstanceHelperMixin' https://review.opendev.org/697539
16:42:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add '_create_server' helper https://review.opendev.org/697540
16:42:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Remove 'api' parameter https://review.opendev.org/697694
17:00:30 efried Thanks gibi, I figured that was probably it but I was having a hard time finding where those responses came from.
17:05:02 openstackgerrit Eric Fried proposed openstack/nova master: Use provider mappings from Placement (mostly) https://review.opendev.org/696992
17:05:02 openstackgerrit Eric Fried proposed openstack/nova master: Restore test_minbw_allocation_placement in nova-next job https://review.opendev.org/697586
17:05:08 efried gibi: that should be good to go now. Thanks for the help.
17:14:14 openstackgerrit Matt Riedemann proposed openstack/nova master: Support RequestSpec in populate_retry https://review.opendev.org/697697
17:25:14 mriedem we need to bump the conductor rpc api to 2.0 but i don't want to do that work
17:25:28 mriedem it would allow removing a lot of legacy filter_properties and request spec cruft from compute and conductor though
17:28:12 gibi efried: thanks, +2
17:28:25 efried thanks gibi
17:29:14 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove dead code from MigrationTask._execute https://review.opendev.org/697699
17:32:38 mriedem efried: so a short spec for https://blueprints.launchpad.net/nova/+spec/ironic-tenant-filter right?
17:32:46 mriedem looks like that was the result of the meeting
17:35:06 openstackgerrit Balazs Gibizer proposed openstack/nova master: Move _update_pci_request_spec_with_allocated_interface_name https://review.opendev.org/696574
17:35:07 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support live migration with qos ports https://review.opendev.org/695905
18:03:44 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Use COMPUTE_SAME_HOST_COLD_MIGRATE trait during migrate https://review.opendev.org/695220
18:03:48 mriedem dansmith: artom: efried: rebased poc ^
18:10:28 efried ack
18:34:27 openstackgerrit Merged openstack/nova stable/stein: compute: Take an instance.uuid lock when rebooting https://review.opendev.org/696152
19:01:00 openstackgerrit Merged openstack/nova master: Use Placement 1.34 (string suffixes & mappings) https://review.opendev.org/696418
19:21:28 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Drop nova-network-base security group tests https://review.opendev.org/696508
19:21:29 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Convert remaining API tests to use neutron https://review.opendev.org/696509
19:21:29 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Convert remaining unit tests to neutron https://review.opendev.org/696510
19:21:30 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove 'is_neutron_security_groups' function https://review.opendev.org/696511
19:21:30 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove nova-network security group driver https://review.opendev.org/696512
19:21:31 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Correct some broken VIF tests https://review.opendev.org/696513
19:21:31 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove firewall support https://review.opendev.org/696514
19:21:32 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Copy shared utils from nova-net module https://review.opendev.org/696515
19:21:32 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove final references to nova-network https://review.opendev.org/696516
19:21:33 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove 'MetadataManager' https://review.opendev.org/696748
19:21:33 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Add TODOs to remove security group-related objects https://review.opendev.org/696517
19:21:34 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove dependency on nova-net from fake cache https://review.opendev.org/696749
19:21:34 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Kill it https://review.opendev.org/696518
19:21:35 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Rename 'nova.network.neutronv2' -> 'nova.network' https://review.opendev.org/696745
19:21:35 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Make the security group API as module https://review.opendev.org/696746
19:21:36 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unnecessary 'neutronv2' prefixes https://review.opendev.org/696776
19:21:36 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove unused exceptions https://review.opendev.org/697149
19:21:37 openstackgerrit Stephen Finucane proposed openstack/nova master: nova-net: Remove db methods for ProviderMethod https://review.opendev.org/697150
19:24:06 openstackgerrit Merged openstack/nova stable/train: Do not update root_device_name during guest config https://review.opendev.org/696339
19:24:23 openstackgerrit Merged openstack/nova master: refactor: RequestGroup.is_empty() and .strip_zeros() https://review.opendev.org/696991
20:18:10 mriedem efried: tox -e fast-specs ftw :)
20:23:20 openstackgerrit Matt Riedemann proposed openstack/nova-specs master: Filter nodes by owner (spec) https://review.opendev.org/697717
20:24:13 mriedem larsks: i added you to ^ so you can keep me honest
20:27:12 openstackgerrit Matt Riedemann proposed openstack/nova stable/stein: Do not update root_device_name during guest config https://review.opendev.org/696351
20:52:30 mriedem gibi: a few questions in https://review.opendev.org/#/c/696246/ for monday
23:05:41 openstackgerrit Merged openstack/nova master: Skip cpu comparison on AArch64 https://review.opendev.org/589769
#openstack-nova - 2019-12-07
00:40:40 efried nice
02:48:06 openstackgerrit XiaojueGuan proposed openstack/nova master: Fix Typo mistake in documentation of "host aggregates in nova" https://review.opendev.org/697745
05:07:39 openstackgerrit Merged openstack/nova stable/rocky: compute: Take an instance.uuid lock when rebooting https://review.opendev.org/696153
06:52:28 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/694717
20:00:28 openstackgerrit Merged openstack/nova stable/queens: compute: Take an instance.uuid lock when rebooting https://review.opendev.org/696154
#openstack-nova - 2019-12-08
15:32:43 openstackgerrit Merged openstack/nova master: Fix Typo mistake in documentation of "host aggregates in nova" https://review.opendev.org/697745
16:44:23 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove 'nova-xvpvncproxy' https://review.opendev.org/687909
#openstack-nova - 2019-12-09
02:52:07 openstackgerrit Sundar Nadathur proposed openstack/nova master: ksa auth conf and client for Cyborg access https://review.opendev.org/631242
02:52:07 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add Cyborg device profile groups to request spec. https://review.opendev.org/631243
02:52:08 openstackgerrit Sundar Nadathur proposed openstack/nova master: Define Cyborg ARQ binding notification event. https://review.opendev.org/692707
02:52:08 openstackgerrit Sundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs. https://review.opendev.org/631244
02:52:09 openstackgerrit Sundar Nadathur proposed openstack/nova master: Compose accelerator PCI devices into VM's domain XML. https://review.opendev.org/631245
02:52:09 openstackgerrit Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735
02:52:10 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999
02:52:10 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable hard reboot with accelerators. https://review.opendev.org/697940
04:02:29 openstackgerrit Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735
04:02:29 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable hard reboot with accelerators. https://review.opendev.org/697940
04:02:30 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999
05:30:04 openstackgerrit Sundar Nadathur proposed openstack/nova master: Define Cyborg ARQ binding notification event. https://review.opendev.org/692707
05:30:04 openstackgerrit Sundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs. https://review.opendev.org/631244
05:30:05 openstackgerrit Sundar Nadathur proposed openstack/nova master: Compose accelerator PCI devices into VM's domain XML. https://review.opendev.org/631245
05:30:05 openstackgerrit Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735
05:30:06 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable hard reboot with accelerators. https://review.opendev.org/697940
05:30:06 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999
06:44:43 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/694717
09:16:29 openstackgerrit Brin Zhang proposed openstack/nova master: PoC: Support re-configure the delete_on_termination in server https://review.opendev.org/693828
11:18:35 slaweq efried: hi
11:18:59 slaweq efried: recently I saw quite often grenade jobs failing with errors like https://02475c780c6fc32e71dc-f63b2c309fc0040fbb4a377b77794f40.ssl.cf1.rackcdn.com/697035/1/check/neutron-grenade-dvr-multinode/baba7e2/logs/testr_results.html.gz
11:19:24 slaweq basically it is "No valid host was found. There are not enough hosts available.'" error from nova in some tests and tests are failing
11:19:51 slaweq efried: did You see something similar? and is there any bug reported for that or should I open new one?
11:23:03 slaweq efried: and in the scheduler log I see something like here: https://zuul.opendev.org/t/openstack/build/baba7e2f78994deabbc3230b3f9acc80/log/logs/screen-n-sch.txt.gz#2505
11:23:43 slaweq and before that there is log like: https://zuul.opendev.org/t/openstack/build/baba7e2f78994deabbc3230b3f9acc80/log/logs/screen-n-sch.txt.gz#2501
11:23:54 slaweq "Timed out waiting for response from cell 258a967d-07ef-43a6-b81e-2ac433a583ef"
11:24:29 slaweq efried: so now I think it's the same issue as https://bugs.launchpad.net/nova/+bug/1844929 - is that correct?
11:24:29 openstack Launchpad bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed]
11:34:27 sean-k-mooney the availabliy zone filter is a red herring that iteration started with 0 hosts https://zuul.opendev.org/t/openstack/build/baba7e2f78994deabbc3230b3f9acc80/log/logs/screen-n-sch.txt.gz#2503
11:41:41 sean-k-mooney slaweq: we do see the same messaging timeout in the n-cpu log on the sub node https://02475c780c6fc32e71dc-f63b2c309fc0040fbb4a377b77794f40.ssl.cf1.rackcdn.com/697035/1/check/neutron-grenade-dvr-multinode/baba7e2/logs/subnode-2/screen-n-cpu.txt.gz
11:44:14 sean-k-mooney slaweq: im also seeing similar error in the agent log
11:45:58 sean-k-mooney it kind of looks like a rabbitmq issue
11:54:45 slaweq sean-k-mooney: rabbitmq timeout happend first time around 19:57 in nova-compute on subnode-2
11:55:05 slaweq in rabbitmq logs it seems that during this time there were only closing connection logs: https://02475c780c6fc32e71dc-f63b2c309fc0040fbb4a377b77794f40.ssl.cf1.rackcdn.com/697035/1/check/neutron-grenade-dvr-multinode/baba7e2/logs/rabbitmq/rabbit%40ubuntu-bionic-ovh-gra1-0013217908.txt.gz

Earlier   Later