Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-11-01
12:51:49 moshele alex_xu: :)
12:52:12 efried jaypipes About this common RequestGroup class definition:
12:52:49 efried jaypipes I get that it appears like code duplication; but realistically, it's not all that uncommon for either side of an API boundary to have very similar or identical objects.
12:53:10 efried In the rest of the nova API we just share the nova.objects or .models stuff on either side.
12:54:09 efried And in fact, if any of the JSON payloads we've been sending back and forth over placement had ever been object-ified, we would have already run into this.
12:54:35 efried Or put the other way: if I had made RequestGroup just a dict, we wouldn't be having this discussion.
12:55:32 efried All this to express my opinion, that it's not at all inappropriate for either side of an API to have substantially identical object representations for the same thing.
12:55:34 jaypipes efried: ack.
12:55:55 efried I suppose if we had os-placement, it could expose the RequestGroup class in a lib and nova could import it.
12:56:15 efried and perhaps it's time to start designing package structure with that in mind.
12:56:46 efried I.e. basically do as cdent said and put it in its own spot, but have that spot be in a path that would ultimately live in an exportable module within the placement codebase.
12:56:48 jaypipes efried: as mentioned earlier, I'm torn on this one :) We actually *used* to use the ResourceProvider *object* within the scheduler client, then I replaced that usage with just a dict representation. Now, we use the ProviderTree in the scheduler client, and we don't use that class in the placement server side. So I get where you're coming from.
12:59:30 efried jaypipes Yeah, I get it.
13:06:32 alex_xu jaypipes: finally, we will build the provider-tree in the placement side for nested support on _get_by_filters(), right?
13:07:04 efried or something like it
13:07:58 jaypipes alex_xu: no, I wasn't planning on doing that. I don't believe we'll need to use the ProviderTree class in the server side. Only the root_provider_id and maybe the parent_provider_id columns of the resource_providers table will be queried.
13:08:53 alex_xu initial I thought we just need to build a set of branches of tree, but that doesn't match some cases efried told me
13:09:36 efried That's going to be a tricky bit of sql for trait propagation.
13:09:44 jaypipes efried: we'll see...
13:09:46 alex_xu jaypipes: ok, sounds like a magic SQL at here
13:48:47 bhagyashris Hi all, Can anyone please guide me using uwsgi how the the different ports are hiding ?
13:48:56 bhagyashris and where exactly the file that maps services to endpoints for uwsgi
13:49:05 bhagyashris for example g-api service is running on 60999 port but when the other services are using the glance like nova then the are using url/image not like url:60999
13:51:23 openstackgerrit Ed Leafe proposed openstack/nova master: Return Selection objects from the scheduler driver https://review.openstack.org/495854
13:51:24 openstackgerrit Ed Leafe proposed openstack/nova master: Change RPC for select_destinations() https://review.openstack.org/516707
13:51:24 openstackgerrit Ed Leafe proposed openstack/nova master: Modify select_destinations() to return objects and alts https://review.openstack.org/510159
13:51:25 openstackgerrit Ed Leafe proposed openstack/nova master: Make conductor pass and use host_lists https://review.openstack.org/511358
13:51:25 openstackgerrit Ed Leafe proposed openstack/nova master: Move the claim_resources method to scheduler utils https://review.openstack.org/511357
13:57:56 openstackgerrit OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/511035
14:17:35 mriedem dansmith: i debugged the CI failure on https://review.openstack.org/#/c/516708/ and it's weird
14:17:47 mriedem it fails because we don't find the instance allocation for some reason http://logs.openstack.org/08/516708/3/gate/legacy-tempest-dsvm-py35/4d8d6a3/logs/screen-n-super-cond.txt.gz#_Oct_31_23_18_04_391235
14:20:35 mriedem i can see the allocations created for that instance here http://logs.openstack.org/08/516708/3/gate/legacy-tempest-dsvm-py35/4d8d6a3/logs/screen-placement-api.txt.gz#_Oct_31_23_18_00_637846
14:20:59 mriedem but shortly after that something deletes the instance allocations http://logs.openstack.org/08/516708/3/gate/legacy-tempest-dsvm-py35/4d8d6a3/logs/screen-placement-api.txt.gz#_Oct_31_23_18_00_805083
14:25:13 efried mriedem Opinion on something?
14:25:38 mriedem ok
14:25:53 efried In the work for granular resource requests, we're finding ourselves needing a very similar POD obj on either side of the placement API barrier.
14:26:23 mriedem what is a POD object?
14:26:31 efried plain ol' data
14:26:49 efried Question is: do we duplicate the base class definition on both sides, define it in one place?
14:27:15 efried If the latter, where would that be, given the long-term goal of separating out the placement API.
14:27:51 mriedem is it a versioned object?
14:28:24 efried No. It doesn't go across the wire. It's just used to represent the same concept on either side.
14:28:26 efried In real terms: https://review.openstack.org/#/c/515151/10/nova/scheduler/utils.py@47 and https://review.openstack.org/#/c/514091/6/nova/api/openstack/placement/util.py@36
14:28:58 efried On the flavor side, it gets built by parsing extra_specs. On the placement side, it gets built by parsing the query string.
14:29:59 mriedem what you have is probably fine isn't it? they aren't exactly identical
14:30:23 mriedem i don't think we're going to create a library for a single object to share between placement and nova right now, maybe when the split happens and there is more common code to share
14:31:15 mriedem there is also nova/common/
14:31:19 efried mriedem Yeah, they're not exactly identical, though they easily could be made so.
14:31:41 mriedem idk, throw them into nova/common/ if you want a single object right now
14:32:07 mriedem nova/common/placement.py?
14:32:22 efried I'm fine keeping them separated - philosophically I would think it's not at all uncommon for identical or almost-identical classes to exist on either side of an API boundary.
14:32:52 efried I mean, if this were e.g. the neutron API we were talking to, we would at best be talking about importing something from a neutron lib.
14:33:55 efried Anyway, jaypipes was on the fence. Sounds like you also don't have a super strong bias either way.
14:35:46 mriedem yeah i don't
14:36:10 openstackgerrit Eric Fried proposed openstack/nova master: placement: Parse granular resources & traits https://review.openstack.org/514091
14:36:25 efried mriedem Thanks.
14:38:11 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Numbered groupings to GET /allocation_candidates https://review.openstack.org/514092
14:42:02 efried jaypipes Just noticed that the filtery part of nova.objects.resource_provider.ResourceProviderList#_get_all_by_filters_from_db is (or should be) common with your _get_provider_ids_matching_all
14:42:23 efried jaypipes (still haven't finished the series, so maybe you already factored that out.)
14:46:47 efried alex_xu Still around?
14:49:11 openstackgerrit Matt Riedemann proposed openstack/nova master: Raise specific exception when swapping migration allocations fails https://review.openstack.org/517004
14:49:25 dansmith mriedem: hmm, you think there's a bug in that delete-if-deleted logic
14:49:38 mriedem dansmith: yes, that's what's deleting the allocation
14:49:42 mriedem http://logs.openstack.org/08/516708/3/gate/legacy-tempest-dsvm-py35/4d8d6a3/logs/screen-n-cpu.txt.gz#_Oct_31_23_18_00_780729
14:49:57 mriedem https://review.openstack.org/517004 fixes the misleading 400 out of the API
14:51:29 mriedem dansmith: so this must be a race between the time the RT tracks the instance as a 'known instance' and the time the update_available_resource periodic runs
14:52:15 dansmith mriedem: it's not supposed to delete it unless it's really deleted=yes though
14:52:16 mriedem yeah the periodic starts here http://logs.openstack.org/08/516708/3/gate/legacy-tempest-dsvm-py35/4d8d6a3/logs/screen-n-cpu.txt.gz#_Oct_31_23_18_00_165850
14:52:41 dansmith mriedem: although, I wonder if it might create the allocation before it's created in the cell db and thus it's a BR only, compute thinks it's deleted like archived
14:52:56 mriedem eesh
14:54:02 mriedem yeah i guess (1) scheduler creates allocation, (2) periodic on compute starts - gets allocations against itself, deletes allocations b/c InstanceNotFound in cell (3) superconductor creates instance in cell for the selected host
14:54:14 mriedem shite
14:54:43 dansmith yeah
14:54:44 mriedem and...we can't really have the compute try to find out if a build request exists can we given that's API DB and the compute shouldn't have access to the API DB
14:54:55 dansmith nope
14:55:01 mriedem i mean, i guess if you're ocata or pike single-cell that might be something you're still doing
14:55:04 mriedem for the late affinity check
14:55:26 mriedem well i'll open a bug to track it anyway
14:55:28 dansmith we've never had computes use objects that are only on the api db, AFAIK,
14:55:41 mriedem server group is in the api db
14:55:45 mriedem *InstanceGroup
14:55:53 dansmith well, because it moved
14:56:08 dansmith anyway, that's a door we don't want to open, even under config I think
14:56:08 mriedem yeah
14:56:32 dansmith deleting only if we find an actually-deleted instance would be one way to handle it,
14:56:43 dansmith which pushes the race to the delete-archive boundary
14:56:48 dansmith but that's much better, IMHO
14:57:17 dansmith if they're doing fast purging (like immediate purging of all deleted things, then they could bump into it, but much less likely, and we could have nova-manage check for allocations potentially
15:01:54 openstackgerrit Matthew Booth proposed openstack/nova master: libvirt: Don't VIR_MIGRATE_NON_SHARED_INC without migrate_disks https://review.openstack.org/507202
15:02:20 openstackgerrit Matthew Booth proposed openstack/nova master: DNM: Run test_volume_backed_live_migration and iscsi test https://review.openstack.org/508163
15:03:31 openstackgerrit Jianghua Wang proposed openstack/nova master: XenAPI: get vGPU stats from hypervisor https://review.openstack.org/512965
15:05:16 jianghuaw jaypipes, thanks for the comment. Yes, you're correct. The above is the reworked revision.
15:06:20 openstackgerrit OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/511035
15:06:22 mriedem dansmith: ok here is the bug https://bugs.launchpad.net/nova/+bug/1729371
15:06:23 openstack Launchpad bug 1729371 in OpenStack Compute (nova) "ResourceTracker races to delete instance allocations before instance is mapped to a cell" [High,Triaged]
15:06:41 dansmith mriedem: roger
15:10:35 mriedem stephenfin: can you take a look at https://review.openstack.org/#/c/481116/ and below?
15:10:52 mriedem i'd like to get that fixed and backported through all supported stable branches since it's a regression since newton
15:11:13 mriedem well, less of a regression than a busted feature since newton for evacuating with a target host
15:12:59 openstackgerrit Matthew Booth proposed openstack/nova master: Remove unused block_migration argument to _live_migration_operation https://review.openstack.org/517007

Earlier   Later