| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-26 | |||
| 10:44:10 | rm_work | it's really a network implementation choice | |
| 10:44:17 | rm_work | gibi: correct | |
| 10:44:31 | rm_work | currently, live-migrate cannot work in a routed-network enabled cloud | |
| 10:44:44 | gibi | rm_work: wooot! Finally I undestood it :) | |
| 10:45:04 | rm_work | because the scheduling doesn't understand it, and will pick some random HV based on other scheduling criteria, and then that HV may not actually be in the correct aggregate for plugging the old port :) | |
| 10:45:57 | gibi | plugging the old port in live migrate case fails becuase the port is already assigned to segment? while in the normal boot case the port is in a deferred state so the plug wont fail | |
| 10:46:04 | rm_work | yes | |
| 10:46:16 | rm_work | live-migrate needs to keep the old address (port) | |
| 10:46:25 | rm_work | a segment is a subnet attribute | |
| 10:46:33 | rm_work | port has an address (which is part of a subnet) | |
| 10:46:54 | sean-k-mooney | gibi: pluging the port on live migration would work if we live migrated to the same netwrok segment | |
| 10:46:56 | rm_work | so one Network can have many Subnets, each which is a separate segment | |
| 10:47:03 | gibi | ok, so during cold migrate if the port needs to be plugged to another segment then it will change IP address but it si not a problem as the guest is rebooted anyhow | |
| 10:47:12 | ileixe | "neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side" <- that's the point that I was unaware of | |
| 10:47:15 | rm_work | sean-k-mooney: right, so that's what we're talking about -- this simple filter allows for that | |
| 10:47:22 | ileixe | Thanks for explanation I will look at it | |
| 10:47:23 | sean-k-mooney | gibi: it should not change ip | |
| 10:47:57 | gibi | sean-k-mooney: but if it changes segment then it will change IP | |
| 10:48:04 | rm_work | I actually haven't even paid attention to cold-migrate, we don't do it either, or care :D it may have the same issue as live-migrate | |
| 10:48:14 | gibi | ahh I see | |
| 10:48:20 | rm_work | I am not sure | |
| 10:48:22 | sean-k-mooney | that my point its not vaild for the port to change its ip on a move operation so it cannot change segment | |
| 10:48:30 | rm_work | right | |
| 10:48:32 | sean-k-mooney | which is why it fails today | |
| 10:48:42 | rm_work | sean-k-mooney: https://review.opendev.org/#/c/709280/ | |
| 10:48:56 | gibi | rm_work: look, I understand you dont care but I as a nova maintainer needs to care. Hence my previous -1 as I did not understood the feature | |
| 10:49:00 | rm_work | this is a very very simple way to get live-migrate and other service-to-service use-cases unblocked | |
| 10:49:26 | sean-k-mooney | we use ip_allocation=defer to allow the ip to be allcoated when we bind the port but it cannoth change again after that point | |
| 10:49:36 | rm_work | gibi: right, sorry, I didn't mean that I don't care if all features work -- i mean i didn't care personally so hadn't checked whether it was the same issue -- this would solve it anyway if it is, and not affect it if it isn't | |
| 10:49:58 | rm_work | i'm not going to propose a feature that breaks other stuff :) | |
| 10:50:20 | gibi | sean-k-mooney: I rely on you about defining what is the expected behavior of a port during cold migrate in the current case so if you say it should not change IP then it accept that | |
| 10:50:42 | rm_work | sure, and in that case, this would allow to solve for cold-migrate as well | |
| 10:50:49 | rm_work | in the same way | |
| 10:51:18 | gibi | rm_work: no worries. I need to act as a guardian not you. I just explained why I was -1 | |
| 10:51:25 | sean-k-mooney | rm_work: just looking at it now but how is the segment id passed to nova? a schduler hint or somehting like that | |
| 10:51:34 | rm_work | sean-k-mooney: scheduler hint, yes | |
| 10:52:12 | sean-k-mooney | --hint routed_segments=96a9316a-54cb-4043-9ccc-b9cacd0d4d52 | |
| 10:52:25 | sean-k-mooney | so is that uuid a placement aggreate uuid | |
| 10:52:37 | sean-k-mooney | so we convert that to a member_of ? | |
| 10:52:41 | gibi | right now I like the simplicity of the filter, but I does not like matching the _name_ of the aggregate as that feels hackish | |
| 10:52:49 | rm_work | sean-k-mooney: this is a neutron segment_id | |
| 10:53:10 | rm_work | gibi: yeah I am not a huge fan of name-comparison as a matcher either, but per the spec it DOES work | |
| 10:53:19 | rm_work | so, this stuff is also in Placement as well | |
| 10:53:27 | rm_work | and in Placement it has a little tighter mapping | |
| 10:53:37 | sean-k-mooney | right so filters are not allowed to call other servcies rest apis so im wondering how nova knows if a host is connected to the segment | |
| 10:53:39 | gibi | rm_work: yeah I got it that this aggregate naming thing was how it is speced and implemented | |
| 10:53:46 | rm_work | sean-k-mooney: nova aggregates | |
| 10:53:55 | gibi | sean-k-mooney: ^^ | |
| 10:54:03 | sean-k-mooney | if we have nova host aggreates we dont need a new filter | |
| 10:54:13 | rm_work | sean-k-mooney: see for example http://paste.openstack.org/show/7YyK5FtL2OiXgErAjMbM/ | |
| 10:54:45 | rm_work | we don't know the correct nova aggregate | |
| 10:54:56 | rm_work | other services would know the segment_id | |
| 10:55:07 | rm_work | nove understands the host-aggregates internally and how they map to segments | |
| 10:55:43 | rm_work | sorry, when I say "we" in this case I mean other services | |
| 10:55:54 | rm_work | (I am speaking from the Octavia point of view, for example) | |
| 10:56:10 | rm_work | What Octavia has is a port, with a subnet and thus a segment_id | |
| 10:56:38 | rm_work | Octavia needs to tell Nova to schedule to the aggregate that has that segment_id | |
| 10:57:37 | rm_work | I guess you're saying that there exists a filter already that would allow us to specify an aggregate for scheduling, and we could just look that up? | |
| 10:58:03 | sean-k-mooney | i think there is one that would allow that yes | |
| 10:58:23 | sean-k-mooney | i tought we could use the json filter for that but it looks like the answer is no | |
| 10:58:59 | gibi | sean-k-mooney: AggregateInstanceExtraSpec filter works based on flavors and aggregate extra spec. but we only have the aggregate name to match | |
| 10:59:03 | rm_work | sean-k-mooney: so here is the example patch for how to make live-migrate work with this: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/ | |
| 10:59:37 | rm_work | gibi: i guess TECHNICALLY it's possible for other services to query nova for aggregate list, and do the matchup on their own | |
| 10:59:44 | sean-k-mooney | gibi: ya the aggreate image and extraspec filters are not the right granulatrity | |
| 10:59:53 | sean-k-mooney | i was considering the compute capabliteis filter | |
| 10:59:57 | rm_work | assuming we could pass an aggregate_id? | |
| 11:00:07 | sean-k-mooney | but setting the segment on each host would be a pain | |
| 11:00:20 | gibi | rm_work: you could not pass agg_id either, you can specify agg extra_spec in the flavor | |
| 11:00:31 | rm_work | in the *flavor*? | |
| 11:00:41 | rm_work | yeah, that's not workable | |
| 11:00:42 | sean-k-mooney | the issue with the filter based approch is if you change the default result set size form placment it can end up retruning only hosts that cant pass the filter | |
| 11:01:02 | rm_work | hmm | |
| 11:01:02 | sean-k-mooney | its the issue that cern had with only 10 results | |
| 11:01:04 | gibi | rm_work: yeah in the flavor, so I getting to grasp why you are in pain | |
| 11:01:15 | sean-k-mooney | /results/allocation_candiates | |
| 11:01:31 | rm_work | yeah we don't want to lock one flavor to one aggregate T_T | |
| 11:01:41 | rm_work | sean-k-mooney: hmm interesting... | |
| 11:02:05 | sean-k-mooney | i dont think we should do this as a filter. we could do it as a pre-filter | |
| 11:02:10 | gibi | sean-k-mooney: that is a valid point, but limit is configurable so the deployer can ask for a lot of candidates via a big limit to avoid this | |
| 11:02:16 | sean-k-mooney | and transfrom the hint into a member_of | |
| 11:02:21 | rm_work | so if we had, say, 100 HVs and 10 segments... it's possible that the filter would only even have 10 hosts as candidates, none of which are in the target filters? | |
| 11:02:22 | gibi | sean-k-mooney: pre-filter is mriedem's approach | |
| 11:02:26 | rm_work | *target segments? | |
| 11:02:51 | gibi | rm_work: the ac limit is 1000 by default but yes | |
| 11:02:56 | rm_work | ahaha ok | |
| 11:03:06 | sean-k-mooney | gibi: yes although i think we need a much more holistic aproch as i said in my comment on his patch | |
| 11:03:43 | gibi | sean-k-mooney: I'm totally up for a holistic approach I just did not understand mriedem's way of doing this yet | |
| 11:03:52 | gibi | but this current discussion will help about that | |
| 11:04:55 | sean-k-mooney | ack | |
| 11:06:25 | gibi | sean-k-mooney: so your approach would be to take the hint and use it as a member_of query. As these aggregates are nova aggregatest they are expected to be mirrored in placement too so member_of would work | |
| 11:06:43 | rm_work | yeah so the point of doing it this way was to try to make it LESS controversial :D | |
| 11:06:51 | sean-k-mooney | well my short term hack yes | |
| 11:07:00 | sean-k-mooney | i dont think we should need a hint | |
| 11:07:02 | rm_work | simplify the use-case target (admin, not users) and make it an optional filter, not in normal code-path | |
| 11:07:04 | gibi | and this way we could avoid the issue with the a_c limit | |
| 11:07:21 | rm_work | so yeah, they're in placement | |
| 11:07:23 | sean-k-mooney | i want neutron to pass the placmeent aggretae and not require the create of any nova aggreates | |
| 11:07:50 | sean-k-mooney | the ip segments should be mirrored by neutron into placement as aggretes | |
| 11:07:59 | gibi | sean-k-mooney: and since qos nova support we have a way to pass traits and RCs but not aggregates from neutron to nova | |
| 11:08:15 | sean-k-mooney | yes so we would have to extend that | |