Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-05
14:49:45 sambetts ?
14:49:47 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Add regression test for rebuilding a volume-backed server https://review.openstack.org/525632
14:49:48 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Get original image_id from volume for volume-backed instance rebuild https://review.openstack.org/525634
14:49:48 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Only query BDMs once in API during rebuild https://review.openstack.org/525633
14:51:19 jaypipes sambetts: I'm referring to the scheduler, not placement. the scheduler checks to see whether a compute node/service is up and available to take requests before it attempts to schedule an instance to it. that check will return False for an Ironic node that is cleaning state.
14:51:23 openstackgerrit Merged openstack/python-novaclient master: Updated from global requirements https://review.openstack.org/525399
14:52:25 sambetts jaypipes: is that new? I've never heard of that before
14:52:54 jaypipes sambetts: so placement might return that Ironic node to the scheduler as having space (now that the allocation was deleted), but the scheduler won't pick it until the node is available.
14:53:06 jaypipes sambetts: no, that's been like that since the beginning.
14:54:06 jaypipes sambetts: https://github.com/openstack/nova/blob/master/nova/scheduler/driver.py#L55-L61
14:54:39 jaypipes though I'm looking at that now and seeing it's referring to the service (i.e. the nova-compute), not the baremetal node
14:54:42 jaypipes ffs
14:54:53 jaypipes this is why we can't ever have nice code...
14:55:09 sambetts :/ never seen anything like that in our driver, its all done based on resources
14:55:13 sambetts yeah :/
14:57:07 jaypipes sambetts: well, there's this patch which should at least help with the Ironic situation: https://review.openstack.org/#/c/513526/
14:57:41 jaypipes sambetts: that randomizes the returned results from placement so that (as is the case with Ironic) you won't always get back the same top node.
14:59:18 sambetts that'll certainly help, the other thing I think we suggested at the PTG was that we somehow instead of the resources getting marked as zero and then removed from placement all the time, we use the reserved field (although I'm still not sure that solves the race)
15:01:58 sambetts the race occurs because the resource tracker is async from the allocation getting deleted, so I'm sure what the right thing to do is there, unless we can force an refresh of that as soon as the resources are freed but then there is still a small period of time while the resource tracker runs
15:03:01 sambetts someone mentioned at the PTG that there are some hypervisors that have the same behaviour as this where even though an allocation has been deleted the resources aren't actually available for use yet
15:03:31 sambetts I'm trying to dig up the notes from the PTG session on it
15:10:10 sambetts hmmm I can't find any notes on the topic in the etherpads :( johnthetubaguy do you remember this conversation at the PTG ^
15:19:00 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285
15:19:00 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Add a new check to volume attach https://review.openstack.org/525622
15:19:01 openstackgerrit Ildiko Vancsa proposed openstack/nova master: libvirt: Allow multiple volume attachments https://review.openstack.org/267587
15:20:47 ildikov mriedem: ^^
15:26:32 mriedem ack
15:44:12 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Add regression test for rebuilding a volume-backed server https://review.openstack.org/525664
15:44:13 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Only query BDMs once in API during rebuild https://review.openstack.org/525665
15:44:14 openstackgerrit Matt Riedemann proposed openstack/nova stable/ocata: Get original image_id from volume for volume-backed instance rebuild https://review.openstack.org/525666
15:48:38 openstackgerrit Ildiko Vancsa proposed openstack/nova master: libvirt: Allow multiple volume attachments https://review.openstack.org/267587
15:48:38 openstackgerrit Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285
16:01:40 edleafe mriedem: time for a question re: alternates for migration?
16:02:54 edleafe if I set return_alternates to True as you suggest, what should then happen to the alternates?
16:03:23 mriedem cold migrate / resize right?
16:03:47 edleafe mriedem: yeah: https://review.openstack.org/#/c/516707/17/nova/conductor/tasks/migrate.py@244
16:04:02 mriedem well, presumably the same thing as happens when build_and_run_instances gets alternates and sends them to compute, which can send them back to build_instances on a reschedule
16:04:42 mriedem so we have 2 reschedule loops:
16:05:11 edleafe well, I'm unfamiliar with those code pathways, so IYO will that be a major change to those methods?
16:05:54 mriedem 2. superconductor:resize_instance -> compute:prep_resize -> cellconductor:resize_instance
16:05:54 mriedem 1. superconductor:schedule_and_build_instances -> compute:build_and_run_instance -> cellconductor:build_instances
16:06:07 mriedem well, looking at https://review.openstack.org/#/c/511358/29/nova/compute/manager.py
16:06:18 mriedem it looks like it would be a matter of passing host_list to prep_resize in the compute
16:07:01 mriedem and like what you have here in conductor manager (this is cell conductor at this point during a reschedule):
16:07:04 mriedem https://review.openstack.org/#/c/511358/29/nova/conductor/manager.py@517
16:07:46 mriedem you would have to do the same for the resize reschedule here https://review.openstack.org/#/c/511358/29/nova/conductor/api.py@87
16:07:53 edleafe ok, next question: should I add that all to the existing patches, or split it into two?
16:08:01 mriedem now, we could arguably do the build and resize + alternates in separate patches
16:08:05 mriedem heh
16:08:14 mriedem split is obviously easier for review
16:08:20 mriedem it would mean 2 compute rpc version bumps
16:08:29 mriedem but service version bumps are free, so i don't think that's a big deal
16:08:33 mriedem dansmith: ^ agree?
16:09:55 edleafe ok, so I'll leave return_alternates=False in the current patch, and then flip it when in a new patch that bumps the compute RPC again.
16:10:07 mriedem makes sense
16:10:13 edleafe ok
16:11:26 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix doubling allocations on rebuild https://review.openstack.org/521662
16:11:28 mriedem jaypipes: ^ is that cve fix (now disclosed) with the bug link and a release note; melwitt can you also look at ^
16:12:40 melwitt mriedem: sure
16:13:09 dansmith mriedem: I like to avoid bumps of both rpc and service version when possible, and it usually is.. but yes, they're free
16:24:28 bauzas mriedem: just +2d the cve fix
16:24:42 mriedem bauzas: thanks
16:24:52 bauzas now the bug is disclosed, we can move on
16:25:13 bauzas sorry for having paid a lot of attention for that bug tho, but the overall direction looked good to me a while ago
16:25:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Fix doubling allocations on rebuild https://review.openstack.org/523214
16:25:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Add regression test for rebuild with new image doubling allocations https://review.openstack.org/523213
16:25:31 bauzas ie. using the hints for passing whether it's a rebuild or not
16:30:04 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Fix doubling allocations on rebuild https://review.openstack.org/523214
16:32:51 mriedem sdague: can you hit https://review.openstack.org/#/c/523194/ again? alex had pointed out something so i lost your +2
16:46:32 openstackgerrit Ed Leafe proposed openstack/nova master: Refactor the code to check for sufficient hosts https://review.openstack.org/520242
16:46:32 openstackgerrit Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239
16:46:33 openstackgerrit Ed Leafe proposed openstack/nova master: Move the to_dict() method to the Selection object https://review.openstack.org/523492
16:46:33 openstackgerrit Ed Leafe proposed openstack/nova master: Return Selection objects from the scheduler driver https://review.openstack.org/495854
16:46:34 openstackgerrit Ed Leafe proposed openstack/nova master: Change RPC for select_destinations() https://review.openstack.org/516707
16:46:34 openstackgerrit Ed Leafe proposed openstack/nova master: Modify select_destinations() to return objects and alts https://review.openstack.org/510159
16:46:35 openstackgerrit Ed Leafe proposed openstack/nova master: Make conductor pass and use host_lists https://review.openstack.org/511358
16:46:35 openstackgerrit Ed Leafe proposed openstack/nova master: Move the claim_resources method to scheduler utils https://review.openstack.org/511357
16:46:39 edleafe mriedem: ^^ addresses your concerns
16:47:11 edleafe mriedem: I'll start on the migrate/resize patch next
16:47:27 jaypipes mriedem: cool, will look shortly, soon as I finish up a rebase of efried_cya_wed's series
16:48:21 mriedem edleafe: ok
17:11:32 melwitt mriedem: now that we're pulling in the dependent os-brick changes, I'm +2 on https://review.openstack.org/#/c/400384
17:32:40 jaypipes cdent: still around?
17:32:53 cdent yessir, still fighting with this grenade stuff
17:33:28 jaypipes cdent: so I don't believe those comments on the API ref thing on the nested resource providers work are correct...
17:33:51 cdent was just reading your responses, one sec
17:34:50 jaypipes I'm also getting sort of annoyed with this :)
17:35:04 cdent jaypipes: I think the confusion is whether what’s listed at /resource_providers is the full rep or not
17:35:17 cdent and up to now it has been: generation, uuid, name, links
17:35:37 cdent https://developer.openstack.org/api-ref/placement/#list-resource-providers
17:35:43 jaypipes no, up until now it's been incorrect. it lists resource_providers (the collective attribute), along with the singular attirbutes
17:36:06 cdent which “it” do you mean?
17:36:21 jaypipes GET /resource_providers response
17:36:48 cdent so you’re saying that for the past 13 microversions what we’ve had at that link above (expand the response example) has been wrong?
17:37:17 jaypipes cdent: I don't see how listing *both* resource_providers (the collective attribute) AND the singular attributes at the same time can be correct.
17:37:38 cdent it’s a list of resource provider objects:
17:37:45 jaypipes yes...
17:37:49 jaypipes and that's all
17:37:55 cdent https://github.com/openstack/nova/blob/master/nova/api/openstack/placement/handlers/resource_provider.py#L110
17:38:15 cdent and the api_ref is set up to list anything, not just the top level things

Earlier   Later