Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-26
10:56:10 rm_work What Octavia has is a port, with a subnet and thus a segment_id
10:56:38 rm_work Octavia needs to tell Nova to schedule to the aggregate that has that segment_id
10:57:37 rm_work I guess you're saying that there exists a filter already that would allow us to specify an aggregate for scheduling, and we could just look that up?
10:58:03 sean-k-mooney i think there is one that would allow that yes
10:58:23 sean-k-mooney i tought we could use the json filter for that but it looks like the answer is no
10:58:59 gibi sean-k-mooney: AggregateInstanceExtraSpec filter works based on flavors and aggregate extra spec. but we only have the aggregate name to match
10:59:03 rm_work sean-k-mooney: so here is the example patch for how to make live-migrate work with this: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/
10:59:37 rm_work gibi: i guess TECHNICALLY it's possible for other services to query nova for aggregate list, and do the matchup on their own
10:59:44 sean-k-mooney gibi: ya the aggreate image and extraspec filters are not the right granulatrity
10:59:53 sean-k-mooney i was considering the compute capabliteis filter
10:59:57 rm_work assuming we could pass an aggregate_id?
11:00:07 sean-k-mooney but setting the segment on each host would be a pain
11:00:20 gibi rm_work: you could not pass agg_id either, you can specify agg extra_spec in the flavor
11:00:31 rm_work in the *flavor*?
11:00:41 rm_work yeah, that's not workable
11:00:42 sean-k-mooney the issue with the filter based approch is if you change the default result set size form placment it can end up retruning only hosts that cant pass the filter
11:01:02 sean-k-mooney its the issue that cern had with only 10 results
11:01:02 rm_work hmm
11:01:04 gibi rm_work: yeah in the flavor, so I getting to grasp why you are in pain
11:01:15 sean-k-mooney /results/allocation_candiates
11:01:31 rm_work yeah we don't want to lock one flavor to one aggregate T_T
11:01:41 rm_work sean-k-mooney: hmm interesting...
11:02:05 sean-k-mooney i dont think we should do this as a filter. we could do it as a pre-filter
11:02:10 gibi sean-k-mooney: that is a valid point, but limit is configurable so the deployer can ask for a lot of candidates via a big limit to avoid this
11:02:16 sean-k-mooney and transfrom the hint into a member_of
11:02:21 rm_work so if we had, say, 100 HVs and 10 segments... it's possible that the filter would only even have 10 hosts as candidates, none of which are in the target filters?
11:02:22 gibi sean-k-mooney: pre-filter is mriedem's approach
11:02:26 rm_work *target segments?
11:02:51 gibi rm_work: the ac limit is 1000 by default but yes
11:02:56 rm_work ahaha ok
11:03:06 sean-k-mooney gibi: yes although i think we need a much more holistic aproch as i said in my comment on his patch
11:03:43 gibi sean-k-mooney: I'm totally up for a holistic approach I just did not understand mriedem's way of doing this yet
11:03:52 gibi but this current discussion will help about that
11:04:55 sean-k-mooney ack
11:06:25 gibi sean-k-mooney: so your approach would be to take the hint and use it as a member_of query. As these aggregates are nova aggregatest they are expected to be mirrored in placement too so member_of would work
11:06:43 rm_work yeah so the point of doing it this way was to try to make it LESS controversial :D
11:06:51 sean-k-mooney well my short term hack yes
11:07:00 sean-k-mooney i dont think we should need a hint
11:07:02 rm_work simplify the use-case target (admin, not users) and make it an optional filter, not in normal code-path
11:07:04 gibi and this way we could avoid the issue with the a_c limit
11:07:21 rm_work so yeah, they're in placement
11:07:23 sean-k-mooney i want neutron to pass the placmeent aggretae and not require the create of any nova aggreates
11:07:50 sean-k-mooney the ip segments should be mirrored by neutron into placement as aggretes
11:07:59 gibi sean-k-mooney: and since qos nova support we have a way to pass traits and RCs but not aggregates from neutron to nova
11:08:15 sean-k-mooney yes so we would have to extend that
11:08:29 sean-k-mooney but that would be next cycle at the earliest
11:08:35 gibi so the full solution needs a neutron change. I guess rm_work will hate that delay :)
11:08:40 sean-k-mooney so if we ignore the approch i would like to do for now
11:08:48 gibi sean-k-mooney: agree
11:09:11 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add framework for extra spec validation https://review.opendev.org/704643
11:09:11 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Add microversion 2.82, extra spec validation https://review.opendev.org/708436
11:09:14 sean-k-mooney i would be semi ok with an experimental pre-filer with the understanding that we will proably remove it next cycle
11:09:19 sean-k-mooney as a FFE
11:09:39 sean-k-mooney if we were to do it as a post filter they are plugable and i dont see why it should be in tree
11:09:54 gibi sean-k-mooney: that pre-filter will be based on a scheduler hint?
11:09:56 sean-k-mooney pre-filters are not plugable so they have to be in tree
11:10:08 sean-k-mooney i think that is the shortest path yes
11:11:03 gibi I see
11:11:15 sean-k-mooney so you dont set ip_allocation defer and precreate the neutron port. neutron assigns it an ip form a segment, then you boot the vm with that port and pass the segment as a hint
11:12:14 sean-k-mooney since the hint will be stored in the request spec it should lock the vm to that segment form then on
11:12:55 sean-k-mooney this would alos work the if you just passed the network to nova and let it create the port
11:13:40 gibi mriedem approach assumes that the segment_id == placement aggregate id so nova cna read the segment id from neutron and construct the placement query without any extra hack
11:14:13 sean-k-mooney yes
11:14:23 gibi that would remove the need of a hint
11:14:40 sean-k-mooney yes
11:14:40 gibi so it would remove the ~ API impact due to the hint
11:15:07 rm_work sean-k-mooney: so, a post-filter that isn't in-tree, can't be a dependency of other services? I think?
11:15:09 sean-k-mooney it does require ues to lookup the segment of every neutron port however
11:15:22 rm_work I mean...
11:15:26 sean-k-mooney rm_work: well not normally no
11:15:38 sean-k-mooney rm_work: what other service are you thinking of
11:15:39 rm_work so, I'm trying to merge code in octavia that would use this
11:15:53 rm_work I linked the change earlier... let me find it again
11:15:57 gibi sean-k-mooney: we do lookup of neutron port information anyhow due to qos so while it is an extra query it is not a totally new thing in the boot and migrate code path
11:15:58 sean-k-mooney well you could package the filter in octavia
11:16:03 sean-k-mooney then i think it would be fine
11:16:13 sean-k-mooney but really im not sure there is time to do this this cycle
11:16:21 rm_work https://review.opendev.org/#/c/706153/
11:17:05 rm_work sean-k-mooney: i was hoping an "optional filter" that was essentially just a few lines would be easier to digest, and we could merge it with less controversy T_T
11:17:47 sean-k-mooney well from a paper work point of view that is not really allowed but being pragmatic maybe. this technical requirs a spec
11:17:59 gibi rm_work: the problem is the new hint, if we merge the filter now, we cannot really remove it later when a better approach is made as the hint becomes an API
11:18:07 rm_work it's fine, we are already running this filter internally (it's very simple to drop-in, as you say) so it's not a big deal. I am trying to unlock some of these features we're using to more folks by putting it upstream
11:18:44 sean-k-mooney rm_work: yep which is nice to see
11:19:13 rm_work for example: we have octavia working in a routed-network setup, and we have live-migrate working
11:19:20 gibi rm_work: and by being here and helping me understand the feature you actually made it a bit more likely that there will be upstream support for this
11:19:27 rm_work sorry, the "we" in this case is actually two orgs, but I've worked for both of them, heh
11:19:56 sean-k-mooney rm_work: for what its worth you could get the same effect by having an availabity zone per ip segment
11:19:59 gibi rm_work: so you made a good step forward here. sorry that this is not that straight forward that is could be. but I think we are in a good track
11:20:01 rm_work (I am working with jroll currently)
11:20:10 rm_work sean-k-mooney: unfortunately that's not an option, heh
11:20:11 sean-k-mooney that is what i know people have done in the past
11:20:23 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Remove native LUKS compat code https://review.opendev.org/669121
11:20:38 openstackgerrit Brin Zhang proposed openstack/nova master: Introduce scope_types in os-volumes-attachments policy https://review.opendev.org/709388
11:21:02 sean-k-mooney so there are a few paths forward.
11:21:34 sean-k-mooney 1.) experimtal prefilter that is disabled by default which would lookup the segment id of each port and add a member of
11:21:53 sean-k-mooney 2.) post filter that woudl use a schduler hint (packaged in octavia)
11:22:04 sean-k-mooney 3.) wait for full solution next cycle
11:22:25 gibi I think I can give some help in 1.)
11:22:29 sean-k-mooney and start on the spec now to have agreement
11:22:31 rm_work well, we'd also like to use the same filter for nova live-migrate, and ironic

Earlier   Later