Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-10-17
16:37:53 mriedem ok then that gets complicated,
16:38:03 sean-k-mooney mriedem: yes you are right sriov or qos implies you precreate the port
16:38:05 mriedem because you can create a server with a nic uuid and nova will create the port in that network
16:38:21 johnthetubaguy that sounds a lot like the routed network case that triggered the move to conductor thingy
16:38:38 mriedem and if the qos policy on the network doesn't match the compute host i guess you have problems
16:38:41 mriedem johnthetubaguy: yes
16:40:15 johnthetubaguy sean-k-mooney: if you want to take on the refactor it is now I think just a case of moving the code, we create missing ports (the conductor bit), then update all the ports (the compute bit)
16:40:16 mriedem the spec also doesn't really mention any details about move operations, which we know are the hairiest thing to deal with
16:41:06 mriedem because during a move the allocations have to be made on the target host and eventually removed from the source host - is the neutron agent going to screwed up by that?
16:41:12 mriedem *to get
16:41:59 sean-k-mooney mriedem: it should not as neutron will not be doing allocations from placement iteself
16:42:30 sean-k-mooney mriedem: but that said we have not gone into details of the move operations in the spec
16:42:52 melwitt stephenfin: I refactored the unit tests to address your comment https://review.openstack.org/#/c/498983
16:43:36 mriedem sean-k-mooney: that was another thing, the spec loosely talked about modifying the nova resource tracker, i think to create allocations for these bandwidth things, but we want the scheduler to create allocations, not the compute (RT)
16:44:09 mriedem because when you move the instance, the source host RT might overwrite the allocations for the instance which are meant for the dest host
16:44:27 mriedem because of periodic task race window wackiness
16:45:25 sean-k-mooney mriedem: yes in the future we defiently want to move the allocation out of the compute node the sepc was assuming the workflow we have today where the compute node does the allocation but i dont think it would break if the allocation was done earlier
16:46:19 mriedem sean-k-mooney: if today is >= pike, we don't create allocatoins in the computes anymore
16:46:31 mriedem allocations in the computes is soooooo 2 months ago :)
16:46:38 sean-k-mooney mriedem: oh did that land in pike cool
16:46:47 mriedem yeah i linked to the change in the spec
16:47:05 mriedem as a result of that change we've been doing a lot of cleanup of places where we dont cleanup allocations
16:47:10 sean-k-mooney mriedem: so is that now done in the conductor/scheduler
16:47:10 mriedem still working on getting those fixes into stable/pike
16:47:18 mriedem the scheduler claims yes
16:47:27 mriedem for VCPUS/MEMORY_MB/DISK_GB
16:47:43 mriedem if you have NUMATopologyLimits, those get claimed in the compute still,
16:47:53 mriedem since we don't model NUMA via nested RPs in placement yet
16:48:18 sean-k-mooney pci device im guessing are still on compute node too for same reason
16:48:40 mriedem yeah
16:49:08 sean-k-mooney to do the claim for the bandwidth in the scheduler we would also have to bind the port in the scheduler
16:49:09 mriedem pci request claims are handled differently from numa limits somehow, i can't remember the details
16:49:37 mriedem sean-k-mooney: why?
16:50:09 mriedem scheduler picks a host, makes the bandwidth allocation against that host, and the port is later bound to the host in the compute
16:50:18 sean-k-mooney to know which network backend to claim the bandwitdh form
16:50:19 mriedem if we fail on the compute, we deallocate
16:50:35 sean-k-mooney e.g. sriov and ovs on same host
16:50:50 mriedem so we don't know the network backend until we bind the host and get a binding:profile from neutron?
16:51:07 sean-k-mooney correct.
16:51:12 mriedem well shit balls
16:51:17 mriedem that's what i call a catch-2
16:51:19 mriedem *catch-22
16:51:43 sean-k-mooney that is why we assumed the current workflow in the spec
16:52:01 mriedem yeah...
16:53:15 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Reproduce bug 1721652 in the functional test env https://review.openstack.org/511759
16:53:17 openstack bug 1721652 in OpenStack Compute (nova) "Evacuate cleanup fails at _delete_allocation_for_moved_instance" [High,In progress] https://launchpad.net/bugs/1721652 - Assigned to Balazs Gibizer (balazs-gibizer)
16:53:17 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: fix cleaning up evacuated instances https://review.openstack.org/512716
16:53:41 sean-k-mooney before ovs offload we could have used the vnic type to differenciate sriov and ovs/linuxbirdge/vpp... but now we cant since vnic type direct is valid for both
16:53:59 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: fix cleaning up evacuated instances https://review.openstack.org/512716
16:54:29 gibi mriedem, bauzas: sorry I was away from the computer. I can do the suggested refactor of https://review.openstack.org/#/c/510938/ in a follow up
16:58:13 sean-k-mooney mriedem: anyway thanks for reviewing it. one spec that is hopefully less contoversial since it was approved last cycle is https://review.openstack.org/#/c/504895/ i think all the code for that has been up for a few months also
17:00:10 sean-k-mooney mriedem: johnthetubaguy oh by the way was this https://review.openstack.org/#/c/499777/5 the cinder attach work you were suggesting emultating for the multiple port binding in relation to how upgrades should work
17:00:27 mriedem sean-k-mooney: no
17:00:35 mriedem sec
17:00:47 mriedem sean-k-mooney: https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/cinder-new-attach-apis.html
17:01:39 sean-k-mooney mriedem: ah ok i have basically added johnthetubaguy suggestions to the latest version but ill go read that now
17:01:42 sean-k-mooney mriedem: thanks
17:02:14 mriedem i need to read the updates on the portbinding spec too
17:02:39 edleafe dansmith: Here's the scenario: cell conductor gets a list of alternates after the initial build attempt fails. Assume that all of these alternates are stale, and no longer have enough resources. Should the cell conductor just log it and do nothing, or should it raise an exception?
17:02:44 edleafe And if an exception, what kind?
17:03:09 mriedem NoValidHost?
17:03:27 mriedem cell conductor raising NoValidHost is something that can already happen today when you run out of alternatives
17:03:30 mriedem it will log a fault
17:03:33 mriedem instance goes to ERROR state
17:03:34 mriedem easy peasy
17:03:49 dansmith edleafe: it should do whatever would happen if we had no retries left in the current path
17:03:50 dansmith yeah
17:03:51 dansmith that
17:03:58 edleafe mriedem: that was my assumption, but I wanted to be sure there wasn't some other action
17:04:07 openstackgerrit Jan Zerebecki proposed openstack/nova master: Fix wording of debug message for future releases https://review.openstack.org/508261
17:04:31 edleafe ah, so that should be the same as what happens in compute when there are no retries left.
17:04:37 dansmith right
17:08:59 openstackgerrit Elod Illes proposed openstack/nova master: WIP: Transform scheduler.select_destinations notification https://review.openstack.org/508506
17:12:35 dansmith edleafe: so, looking at your third patch,
17:12:43 dansmith edleafe: we have limits in the reqspec,
17:13:03 dansmith oh,
17:13:22 dansmith the selection limits are really just for the return value from the rpc call to continue to include them is that right?
17:13:35 dansmith I was thinking we had to have them for communicating to the compute, but that is in reqspec
17:13:37 dansmith is that right?
17:13:57 edleafe dansmith: the caching scheduler needs them
17:14:16 dansmith the caching scheduler _sets_ them
17:14:22 dansmith the compute _needs_ them
17:14:44 edleafe ok, I thought it was needed in the filters
17:15:02 dansmith my point is, we really just need them in Selection so that the return from select_destinations(), which is now a bunch of Selections, can include the limits back to conductor,
17:15:06 edleafe but yeah, the compute will need them too
17:15:11 dansmith which will populate the reqspec
17:15:22 dansmith the filters don't need them, the filters are what put things into limits
17:15:42 edleafe ok, I guess I misunderstood mriedem's concerns
17:15:49 dansmith compute can get them from the reqspec
17:18:44 mriedem the limits are stored in the reqspec?
17:19:01 mriedem tbc, we pass a reqspec down to build_and_run_instance in compute and it's *totally* ignored
17:19:05 mriedem we pass the limits dict separately
17:19:57 mriedem ok i see the SchedulerLimits field in the request spec
17:20:02 mriedem i just don't think compute is using that today
17:20:14 dansmith mriedem: right, so this will be passing limits N more times than we currently pass it
17:20:39 dansmith I'm thinking maybe we just have select_destinations return (reqspec, alternates) with reqspec updated with limits,
17:20:53 dansmith and then we can avoid putting them into the selection
17:21:10 dansmith aside from numa, they'll be the same for num_instances, right?
17:21:14 mriedem need to see what actually populates RequestSpec.limits today
17:21:26 dansmith I think it's populate_filter_properties

Earlier   Later