Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-26
10:49:00 rm_work this is a very very simple way to get live-migrate and other service-to-service use-cases unblocked
10:49:26 sean-k-mooney we use ip_allocation=defer to allow the ip to be allcoated when we bind the port but it cannoth change again after that point
10:49:36 rm_work gibi: right, sorry, I didn't mean that I don't care if all features work -- i mean i didn't care personally so hadn't checked whether it was the same issue -- this would solve it anyway if it is, and not affect it if it isn't
10:49:58 rm_work i'm not going to propose a feature that breaks other stuff :)
10:50:20 gibi sean-k-mooney: I rely on you about defining what is the expected behavior of a port during cold migrate in the current case so if you say it should not change IP then it accept that
10:50:42 rm_work sure, and in that case, this would allow to solve for cold-migrate as well
10:50:49 rm_work in the same way
10:51:18 gibi rm_work: no worries. I need to act as a guardian not you. I just explained why I was -1
10:51:25 sean-k-mooney rm_work: just looking at it now but how is the segment id passed to nova? a schduler hint or somehting like that
10:51:34 rm_work sean-k-mooney: scheduler hint, yes
10:52:12 sean-k-mooney --hint routed_segments=96a9316a-54cb-4043-9ccc-b9cacd0d4d52
10:52:25 sean-k-mooney so is that uuid a placement aggreate uuid
10:52:37 sean-k-mooney so we convert that to a member_of ?
10:52:41 gibi right now I like the simplicity of the filter, but I does not like matching the _name_ of the aggregate as that feels hackish
10:52:49 rm_work sean-k-mooney: this is a neutron segment_id
10:53:10 rm_work gibi: yeah I am not a huge fan of name-comparison as a matcher either, but per the spec it DOES work
10:53:19 rm_work so, this stuff is also in Placement as well
10:53:27 rm_work and in Placement it has a little tighter mapping
10:53:37 sean-k-mooney right so filters are not allowed to call other servcies rest apis so im wondering how nova knows if a host is connected to the segment
10:53:39 gibi rm_work: yeah I got it that this aggregate naming thing was how it is speced and implemented
10:53:46 rm_work sean-k-mooney: nova aggregates
10:53:55 gibi sean-k-mooney: ^^
10:54:03 sean-k-mooney if we have nova host aggreates we dont need a new filter
10:54:13 rm_work sean-k-mooney: see for example http://paste.openstack.org/show/7YyK5FtL2OiXgErAjMbM/
10:54:45 rm_work we don't know the correct nova aggregate
10:54:56 rm_work other services would know the segment_id
10:55:07 rm_work nove understands the host-aggregates internally and how they map to segments
10:55:43 rm_work sorry, when I say "we" in this case I mean other services
10:55:54 rm_work (I am speaking from the Octavia point of view, for example)
10:56:10 rm_work What Octavia has is a port, with a subnet and thus a segment_id
10:56:38 rm_work Octavia needs to tell Nova to schedule to the aggregate that has that segment_id
10:57:37 rm_work I guess you're saying that there exists a filter already that would allow us to specify an aggregate for scheduling, and we could just look that up?
10:58:03 sean-k-mooney i think there is one that would allow that yes
10:58:23 sean-k-mooney i tought we could use the json filter for that but it looks like the answer is no
10:58:59 gibi sean-k-mooney: AggregateInstanceExtraSpec filter works based on flavors and aggregate extra spec. but we only have the aggregate name to match
10:59:03 rm_work sean-k-mooney: so here is the example patch for how to make live-migrate work with this: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/
10:59:37 rm_work gibi: i guess TECHNICALLY it's possible for other services to query nova for aggregate list, and do the matchup on their own
10:59:44 sean-k-mooney gibi: ya the aggreate image and extraspec filters are not the right granulatrity
10:59:53 sean-k-mooney i was considering the compute capabliteis filter
10:59:57 rm_work assuming we could pass an aggregate_id?
11:00:07 sean-k-mooney but setting the segment on each host would be a pain
11:00:20 gibi rm_work: you could not pass agg_id either, you can specify agg extra_spec in the flavor
11:00:31 rm_work in the *flavor*?
11:00:41 rm_work yeah, that's not workable
11:00:42 sean-k-mooney the issue with the filter based approch is if you change the default result set size form placment it can end up retruning only hosts that cant pass the filter
11:01:02 rm_work hmm
11:01:02 sean-k-mooney its the issue that cern had with only 10 results
11:01:04 gibi rm_work: yeah in the flavor, so I getting to grasp why you are in pain
11:01:15 sean-k-mooney /results/allocation_candiates
11:01:31 rm_work yeah we don't want to lock one flavor to one aggregate T_T
11:01:41 rm_work sean-k-mooney: hmm interesting...
11:02:05 sean-k-mooney i dont think we should do this as a filter. we could do it as a pre-filter
11:02:10 gibi sean-k-mooney: that is a valid point, but limit is configurable so the deployer can ask for a lot of candidates via a big limit to avoid this
11:02:16 sean-k-mooney and transfrom the hint into a member_of
11:02:21 rm_work so if we had, say, 100 HVs and 10 segments... it's possible that the filter would only even have 10 hosts as candidates, none of which are in the target filters?
11:02:22 gibi sean-k-mooney: pre-filter is mriedem's approach
11:02:26 rm_work *target segments?
11:02:51 gibi rm_work: the ac limit is 1000 by default but yes
11:02:56 rm_work ahaha ok
11:03:06 sean-k-mooney gibi: yes although i think we need a much more holistic aproch as i said in my comment on his patch
11:03:43 gibi sean-k-mooney: I'm totally up for a holistic approach I just did not understand mriedem's way of doing this yet
11:03:52 gibi but this current discussion will help about that
11:04:55 sean-k-mooney ack
11:06:25 gibi sean-k-mooney: so your approach would be to take the hint and use it as a member_of query. As these aggregates are nova aggregatest they are expected to be mirrored in placement too so member_of would work
11:06:43 rm_work yeah so the point of doing it this way was to try to make it LESS controversial :D
11:06:51 sean-k-mooney well my short term hack yes
11:07:00 sean-k-mooney i dont think we should need a hint
11:07:02 rm_work simplify the use-case target (admin, not users) and make it an optional filter, not in normal code-path
11:07:04 gibi and this way we could avoid the issue with the a_c limit
11:07:21 rm_work so yeah, they're in placement
11:07:23 sean-k-mooney i want neutron to pass the placmeent aggretae and not require the create of any nova aggreates
11:07:50 sean-k-mooney the ip segments should be mirrored by neutron into placement as aggretes
11:07:59 gibi sean-k-mooney: and since qos nova support we have a way to pass traits and RCs but not aggregates from neutron to nova
11:08:15 sean-k-mooney yes so we would have to extend that
11:08:29 sean-k-mooney but that would be next cycle at the earliest
11:08:35 gibi so the full solution needs a neutron change. I guess rm_work will hate that delay :)
11:08:40 sean-k-mooney so if we ignore the approch i would like to do for now
11:08:48 gibi sean-k-mooney: agree
11:09:11 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add microversion 2.82, extra spec validation https://review.opendev.org/708436
11:09:11 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add framework for extra spec validation https://review.opendev.org/704643
11:09:14 sean-k-mooney i would be semi ok with an experimental pre-filer with the understanding that we will proably remove it next cycle
11:09:19 sean-k-mooney as a FFE
11:09:39 sean-k-mooney if we were to do it as a post filter they are plugable and i dont see why it should be in tree
11:09:54 gibi sean-k-mooney: that pre-filter will be based on a scheduler hint?
11:09:56 sean-k-mooney pre-filters are not plugable so they have to be in tree
11:10:08 sean-k-mooney i think that is the shortest path yes
11:11:03 gibi I see
11:11:15 sean-k-mooney so you dont set ip_allocation defer and precreate the neutron port. neutron assigns it an ip form a segment, then you boot the vm with that port and pass the segment as a hint
11:12:14 sean-k-mooney since the hint will be stored in the request spec it should lock the vm to that segment form then on
11:12:55 sean-k-mooney this would alos work the if you just passed the network to nova and let it create the port
11:13:40 gibi mriedem approach assumes that the segment_id == placement aggregate id so nova cna read the segment id from neutron and construct the placement query without any extra hack
11:14:13 sean-k-mooney yes
11:14:23 gibi that would remove the need of a hint
11:14:40 gibi so it would remove the ~ API impact due to the hint
11:14:40 sean-k-mooney yes
11:15:07 rm_work sean-k-mooney: so, a post-filter that isn't in-tree, can't be a dependency of other services? I think?
11:15:09 sean-k-mooney it does require ues to lookup the segment of every neutron port however
11:15:22 rm_work I mean...
11:15:26 sean-k-mooney rm_work: well not normally no
11:15:38 sean-k-mooney rm_work: what other service are you thinking of

Earlier   Later