Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-03-20
15:42:56 belmoreira it iterates per cell but is not possible to select the cell
15:43:44 dansmith ah okay
15:51:38 mnaser belmoreira: changed hostname or compute node by any chance ?
15:55:32 belmoreira mnaser maybe... I'm configuring "conductor_groups" for ironic. To avoid this precise issue I configured only one conductor group to keep the same nova-compute... and move from there. Still looking... for now I don't understand what went wrong
15:55:58 mnaser belmoreira: maybe check the compute nodes table
15:56:42 belmoreira mnaser the compute_nodes table had all the resources deleted
15:59:12 belmoreira mnaser to recover I made the entries available again (deleted=0, deleted_at=NULL)
15:59:36 belmoreira new entries are not created because the uuid conflict
15:59:49 mnaser belmoreira: pretty sure you might look at the compute logs and see that it was deleting them
16:00:07 mnaser also I think I ran into that and mriedem fixed it at the time
16:01:14 belmoreira yeah, I need to go deep into the logs... for now I'm still adding the nodes into the right placement_aggregate for them to be available
16:36:19 mriedem see that link above - if you're doing rebalancing of nodes when the old compute sees the nodes are no longer reported there it deletes them along with the providers and their allocations
16:36:51 belmoreira thanks mriedem
16:36:59 mriedem https://review.opendev.org/#/c/678100/3/nova/compute/manager.py@9208
16:37:09 mriedem that cascading delete thing was added back when the computes still self-healed the allocations in the RT
16:37:15 mriedem but that self-heal was removed a few releases ago
16:37:21 mriedem so now we just delete your stuff on re-balance
16:37:26 mriedem you're welcome
16:38:54 mriedem belmoreira: i think this is the todo you're looking for for heal_allocations yes? https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L2119
16:39:36 mriedem that's a very easy change https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L2173
16:41:33 belmoreira mriedem yes, it's easy to do... I did't under stress and just ran it :)
16:42:05 belmoreira we have a new member in the team, this is a good low hanging fruit for him
17:16:01 openstackgerrit Merged openstack/nova master: Introduce scope_types in os-flavor-access https://review.opendev.org/713559
18:11:03 openstackgerrit melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478
18:11:05 openstack bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929
18:32:17 openstackgerrit melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478
18:32:18 openstack bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929
18:34:56 openstackgerrit melanie witt proposed openstack/nova stable/train: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/713116
18:34:57 openstack bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929
18:49:24 openstackgerrit Artom Lifshitz proposed openstack/nova master: func tests: move _run_periodics() into own mixin https://review.opendev.org/705545
18:49:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
18:49:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: tests: work around malformed serial XML https://review.opendev.org/705546
18:49:26 openstackgerrit Artom Lifshitz proposed openstack/nova master: NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404
18:49:27 openstack bug 1845146 in OpenStack Compute (nova) train "NUMA aware live migration failed when vCPU pin set" [High,Fix committed] https://launchpad.net/bugs/1845146 - Assigned to Dan Smith (danms)
19:03:40 artom dansmith, ^^ if you have the time and energy this Friday afternoon. Next week is fine too :) I'm going to go interact with my spawns for a bit
19:30:24 openstackgerrit Artom Lifshitz proposed openstack/nova master: func tests: move _run_periodics() into own mixin https://review.opendev.org/705545
19:30:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
19:30:25 openstackgerrit Artom Lifshitz proposed openstack/nova master: tests: work around malformed serial XML https://review.opendev.org/705546
19:30:26 openstackgerrit Artom Lifshitz proposed openstack/nova master: NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404
19:30:28 openstack bug 1845146 in OpenStack Compute (nova) train "NUMA aware live migration failed when vCPU pin set" [High,Fix committed] https://launchpad.net/bugs/1845146 - Assigned to Dan Smith (danms)
19:50:03 artom dansmith, so, you'd rather the class inheriting from the Mixin sets self.computes?
19:50:34 artom My thinking was, the mixin is a batteries-included thing to help you run_periodics()
19:50:56 dansmith artom: I'd rather not have the mixin at all, but yes, I'd rather the classes set them and let that blow up if they don't instead effectively delegating setUp() to a later time, and in a mixin
19:51:01 dansmith I understand
19:52:17 artom dansmith, you mean move it directly under _IntegratedTestBase? Not everything can inherit from that...
19:52:41 artom Can it?
19:53:43 dansmith I dunno, you tell me.. I didn't go look at the whole inheritance tree.
19:54:33 artom Yeah, looking now
19:54:36 dansmith the massive mess of where and how all those tests get their functions make it super confusion to figure out where stuff is defined, which is why I don't like this
19:55:10 dansmith the number of places we have duplicated helper functions for checking server state, waiting for server state to change, server state to become a thing, etc is just maddening
19:57:19 artom dansmith, I know - stephenfin's done some work to improve that
19:57:54 artom So, I checked and the two regression tests that I changed would be a a PITA to make inherit from the _IntegratedTestBase
19:58:02 artom They have their own lengthy setup
19:59:16 artom The mixin is probably cleaner, that at point. But OK, let's make the child setup self.computes then
19:59:20 artom *at that point
19:59:42 dansmith start_service is a method on the *base* TestCase right?
19:59:48 dansmith so why not just put all of it there?
19:59:51 artom The very base, yeah
19:59:59 dansmith if that's where they meet in the inheritance tree, that's ...where they meet
20:00:30 artom I suppose
20:00:47 artom Isn't it weird to have func test-specific stuff in the base class?
20:01:03 dansmith like what's already there you mean?
20:01:13 artom Heh, yeah :/
20:01:33 dansmith you could also put another layer in between the two.. _IntegratedTestBaseNoSeriouslyThisIsTheFunctionalBaseForReals
20:01:41 artom lol
21:02:31 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
21:02:31 openstackgerrit Artom Lifshitz proposed openstack/nova master: tests: work around malformed serial XML https://review.opendev.org/705546
21:02:32 openstackgerrit Artom Lifshitz proposed openstack/nova master: NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404
21:02:34 openstackgerrit Artom Lifshitz proposed openstack/nova master: func tests: move _run_periodics() into base class https://review.opendev.org/714228
21:02:34 openstack bug 1845146 in OpenStack Compute (nova) train "NUMA aware live migration failed when vCPU pin set" [High,Fix committed] https://launchpad.net/bugs/1845146 - Assigned to Dan Smith (danms)
22:09:07 openstackgerrit Artom Lifshitz proposed openstack/nova master: NUMA LM: Add func test for bug 1845146 https://review.opendev.org/687404
22:09:07 openstackgerrit Artom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595
22:09:09 openstack bug 1845146 in OpenStack Compute (nova) train "NUMA aware live migration failed when vCPU pin set" [High,Fix committed] https://launchpad.net/bugs/1845146 - Assigned to Dan Smith (danms)
#openstack-nova - 2020-03-21
04:58:28 openstackgerrit Merged openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470
06:08:12 openstackgerrit Brin Zhang proposed openstack/nova master: Make serialize_args handle exception messages safely https://review.opendev.org/712697
06:08:35 openstackgerrit Brin Zhang proposed openstack/nova master: Store instance action event exc_val fault details https://review.opendev.org/694428
06:09:10 openstackgerrit Brin Zhang proposed openstack/nova master: Expose instance action event details out of the API https://review.opendev.org/694430
06:09:26 openstackgerrit Brin Zhang proposed openstack/nova master: Add instance actions v283 samples test https://review.opendev.org/706251
08:49:22 openstackgerrit Merged openstack/nova master: Add config option for neutron client retries https://review.opendev.org/712226
10:26:06 openstackgerrit chenker proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999
17:08:47 openstackgerrit Lee Yarwood proposed openstack/nova master: nova-live-migration: Only stop n-cpu and q-agt during evacuation testing https://review.opendev.org/714057
19:54:45 hrw when machine starts on aarch64 then hw_firmware_type=UEFI is set in code by default (since queens). It is not stored in image metadata at all (when image is checked with 'image show IMAGENAME'). but when I start such image and it fails to start then it looks like nova does not set flag to remove nvram. I looked at libvirt/driver.py and it looks like it is assumed that instance/image will have
19:54:51 hrw hw_firmware_type=UEFI set ;(
19:55:07 hrw [instance: 09554b80-bb49-47bb-bbd3-36d651ab4655] Instance failed to spawn: libvirt.libvirtError: Requested operation is not valid: cannot undefine domain with nvram
20:02:47 hrw yep.
20:03:03 hrw added hw_firmware_type=uefi to the image and it fixed issue
20:03:07 hrw meh ;(
21:21:14 openstackgerrit Merged openstack/nova stable/rocky: Replace ansible --sudo with --become in live_migration/hooks scripts https://review.opendev.org/713839
21:21:21 openstackgerrit Merged openstack/nova stable/rocky: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713840
#openstack-nova - 2020-03-22
01:31:57 openstackgerrit Sundar Nadathur proposed openstack/nova master: ksa auth conf and client for Cyborg access https://review.opendev.org/631242
01:31:58 openstackgerrit Sundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs. https://review.opendev.org/631244
01:31:58 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add Cyborg device profile groups to request spec. https://review.opendev.org/631243
01:31:59 openstackgerrit Sundar Nadathur proposed openstack/nova master: Compose accelerator PCI devices into domain XML in libvirt driver. https://review.opendev.org/631245
01:31:59 openstackgerrit Sundar Nadathur proposed openstack/nova master: Pass accelerator requests to each virt driver from compute manager. https://review.opendev.org/698581
01:32:00 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable hard/soft reboot with accelerators. https://review.opendev.org/697940
01:32:00 openstackgerrit Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735
01:32:01 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable and use COMPUTE_ACCELERATORS trait. https://review.opendev.org/699554
01:32:01 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable start/stop of instances with accelerators. https://review.opendev.org/699553
01:32:02 openstackgerrit Sundar Nadathur proposed openstack/nova master: Block unsupported instance operations with accelerators. https://review.opendev.org/674726
01:32:02 openstackgerrit Sundar Nadathur proposed openstack/nova master: Bump compute rpcapi version and reduce Cyborg calls. https://review.opendev.org/704227

Earlier   Later