Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-02-15
20:48:30 fried_rice tssurya: Like what if ironic actually *wanted* to dork with more than one node at a time?
20:53:00 tssurya fried_rice: since we are talking about why that code exists, can you give me a ore concrete example of when ironic would want that ?
20:53:21 tssurya like I am trying to understand more use cases for this whole thing
20:53:45 tssurya since I am pretty sure we don't need them for now right ?
20:53:54 tssurya or maybe I am missing some documentation
20:54:33 fried_rice tssurya: Yeah, I don't have a specific use case; I could only contrive hypothetical ones. It's just that update_provider_tree is a generic method which passes the *whole* ProviderTree object down to the virt driver for modification, and then update_from_provider_tree is responsible for flushing any changes back to placement.
20:54:50 fried_rice I really didn't want upt/ufpt to be tightly bound to specific virt drivers' implementations or needs in that regard.
20:55:11 fried_rice and it's specifically designed to be future-looking to when we have nested providers, sharing providers, etc.
20:55:24 fried_rice ...just like the code that broke your Queens performance :)
20:55:37 tssurya yea I get that, its just that for ironic I don't know how this would scale with the more nodes that we would keep adding in future
20:55:38 fried_rice ...which wasn't actually going to be used until... I think stein actually.
20:55:56 cdent mriedem: i'll either get to it or get gary, but yeah
20:55:59 fried_rice tssurya: Well, let's see how this does.
20:56:35 fried_rice tssurya: Note that there are already a number of anticipatory optimizations in that code.
20:56:53 fried_rice tssurya: Like the fact that we do local compares and only call out to placement if something has changed.
20:57:20 openstackgerrit melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351
20:57:20 openstackgerrit melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350
20:57:27 tssurya fried_rice: yea I mean as far as most of the things we do right now the flushes to placement are very less
20:57:29 fried_rice tssurya: I just didn't anticipate that iterating through a list would bring the service to its knees. Even if you had told me the list was going to be a couple thousand long.
20:58:43 tssurya fried_rice: of course yea we didn't know this as well until we timed this, our test env is not exactly the size of our actual prod env
21:03:23 openstackgerrit Matt Riedemann proposed openstack/nova master: Check hosts have no instances for AZ rename https://review.openstack.org/509206
21:03:32 mriedem sounds like someone needs their nova-computes running on s390x mainframes
21:03:39 mriedem or Power9!
21:04:34 tssurya fried_rice: just saw your patch, so basically you are saving on the lookup time instead of iterating.. We will try this first thing Monday and let you know
21:04:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Check hosts have no instances for AZ rename https://review.openstack.org/509206
21:05:21 fried_rice tssurya: Yes. We already have the list of UUIDs. Now instead of that loop taking O(N) to look up each node's provider object, it'll take O(1).
21:05:58 tssurya yeap nice
21:06:46 fried_rice tssurya: now, it's possible that that lookup wasn't the problem: it's possible that what's killing us is 1700 ProviderTree.data() calls (which copies the guts of the ProviderTree) plus 5100 compares of that information.
21:06:58 fried_rice that may be a tad harder to optimize.
21:08:08 fried_rice (...guts of the individual _Provider in the ProviderTree, that is)
21:15:18 tssurya yea you mean this part here (https://github.com/openstack/nova/blob/880327cc31fea7328d23355730d5458f3b74662b/nova/scheduler/client/report.py#L1440)
21:15:35 tssurya and then the set aggregates and inventories comparisions
21:24:34 melwitt mriedem: if I'm trying to remove the console-auth workaround in stein and nova-status upgrade check has the check for console auths, where it warns if compute services older than rocky are found and advises to set the [workarounds] option if so. if I'm removing the workaround, it seems like I should remove the upgrade check. is that right?
21:30:17 mriedem does the upgrade check just warn today?
21:30:23 melwitt yes
21:30:25 mriedem and is that upgrade check in stable/rocky?
21:30:45 mriedem i think you'd either remove the upgrade check in stein or change it to a hard failure
21:30:54 mriedem and then maybe mark it for removal in Train
21:31:05 melwitt yes the check is in stable/rocky
21:31:13 openstackgerrit Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.openstack.org/635669
21:31:13 openstackgerrit Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827
21:31:14 openstackgerrit Artom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828
21:31:14 openstackgerrit Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606
21:31:15 openstackgerrit Artom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source https://review.openstack.org/635229
21:31:42 mriedem off the top of my head i think i'd go conservative and make the upgrade check a hard failure in stein and remove it in train
21:31:44 melwitt ok, I considered changing it to a hard failure if any computes older than rocky are found. I guess the action would be "upgrade your computes
21:31:55 mriedem because someone could run the stein upgrade checks against a rocky deployment from a venv/container
21:32:08 mriedem dansmith might have an opinion
21:32:15 melwitt I was struggling a bit with what the action item is for something that generic. all of the other failures say, "run this command"
21:32:37 mriedem yeah i think the only recent upgrade check we removed was the one for placement resource providers because of the extracted placement
21:32:55 mriedem we could have re-worked that to hit the placement API rather than the nova_api db, but it had been around since ocata so figured it was ok to just drop it
21:33:06 mriedem this consoleauth one seems to have really confused some people and caught them off guard
21:33:19 mriedem so dropping the check with the workaround in the same release might be too aggressive
21:34:11 melwitt I wondered that too, because of how problematic it's been. whether to just keep the workaround around another cycle
21:34:49 mriedem if it's not hurting anything that might be fine
21:34:55 mriedem safer than sorryier
21:35:12 melwitt I was thinking to follow through with TODOs before they get too old and forgettable but this seems like one of those where it's best not
21:35:25 melwitt yeah, it doesn't hurt anything
21:35:36 melwitt ok, well, that's easy then
21:37:03 mriedem we've still got lots of todos to remove things from several releases ago
21:37:16 mriedem hell we've still got nova-net and cells v1
21:38:54 melwitt oh I know. I was thinking instead, I should propose one of the old TODOs. I think the instance group migration to API database that I worked on is still around :(
21:41:04 melwitt oh, looks like you removed that last year. heh
21:59:15 mriedem you still have to write the API side changes for the counting quotas from placement thing right?
21:59:18 mriedem s/quotas/usage/
22:11:21 melwitt mriedem: right. I'm trying to debug why my online data migration does an infinite loop in grenade, at the moment :(
22:23:52 mriedem looks like you're mostly just copying populate_queued_for_delete
22:24:04 mriedem the only major difference i see is you're using filter_by and that other uses filter()
22:24:37 mriedem that shouldn't really matter though
22:26:34 melwitt yeah. I did that because I was thinking that's a better approach than what I was originally planning, to get all instance mappings where user_id=None and then lookup cell by project_id and get instance
22:27:58 melwitt tssurya mentioned to me that when she was working on queued_for_delete, the loop happened for that because originally she didn't have a default for InstanceMapping.create(), so it was creating NULL fields that would get found again by the migration (creates must have been happening also in the online data migrations, I guess)
22:29:13 melwitt so I added a default for create(), also added setting of user_id for save() in conductor, in case any instance builds are in-flight during an upgrade, where instance mapping was created with cell=None before upgrade and then before scheduling during upgrade, it gets scheduled, set the user_id
22:29:40 melwitt I don't know what case grenade is hitting. it's looping not in the initial online migration run before starting services, but in the second online migration after starting services
22:30:00 melwitt I added a print statement to the online migration to help with debugging in the next gate run
22:30:21 mriedem i don't see anything obviously wrong
22:30:34 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.openstack.org/637058
22:30:35 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize conductor RPC method https://review.openstack.org/637075
22:30:35 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.openstack.org/637070
22:30:36 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Confirm cross-cell resize from the API https://review.openstack.org/637316
22:30:38 mriedem dansmith: and we've got confirm ^
22:30:39 melwitt thanks for taking a look
22:30:44 mriedem plus functional
22:41:09 melwitt I think I need to take a break
#openstack-nova - 2019-02-16
05:52:14 openstackgerrit Merged openstack/nova master: Cleanup inflight rpc messages between test cases https://review.openstack.org/637233
05:56:34 openstackgerrit Merged openstack/nova master: Plumbing required in servers ViewBuilder to construct partial results https://review.openstack.org/635146
05:56:42 openstackgerrit Merged openstack/nova master: Add context.target_cell() stub to DownCellFixture https://review.openstack.org/637182
11:27:02 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.openstack.org/576712
15:30:16 openstackgerrit Merged openstack/python-novaclient master: add python 3.7 unit test job https://review.openstack.org/637290
18:55:26 openstackgerrit Merged openstack/nova master: Trim fake_deserialize_context in test_conductor https://review.openstack.org/635859
19:30:13 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Ignore VolumeAttachmentNotFound exception in compute.manager https://review.openstack.org/637391
#openstack-nova - 2019-02-17
00:01:33 openstackgerrit Merged openstack/nova master: API microversion 2.69: Handles Down Cells https://review.openstack.org/591657
10:35:54 openstackgerrit Hamdy Khader proposed openstack/os-vif master: Add create_ovs_port field in VIFPortProfileOpenVSwitch profile https://review.openstack.org/636061
#openstack-nova - 2019-02-18
00:39:56 openstackgerrit Artom Lifshitz proposed openstack/nova master: Remove _legacy_dict methods https://review.openstack.org/636210
00:39:57 openstackgerrit Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827
00:39:57 openstackgerrit Artom Lifshitz proposed openstack/nova master: Add migration param to check_can_live_migrate_destination https://review.openstack.org/634605
00:39:58 openstackgerrit Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606
00:39:58 openstackgerrit Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.openstack.org/635669
00:39:59 openstackgerrit Artom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source https://review.openstack.org/635229
00:39:59 openstackgerrit Artom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828
01:58:53 openstackgerrit Zhenyu Zheng proposed openstack/nova master: Add method to allow reset fields for root bdm in BDM obj https://review.openstack.org/614672

Earlier   Later