| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-26 | |||
| 10:37:17 | gibi | I have to read up | |
| 10:37:21 | rm_work | gibi: so for the octavia case, the issue is that we need both of the service-vms we build to be in the same segment as the VIP (an unbound neutron port) we create | |
| 10:37:37 | rm_work | so, we simply look at that port, then look at the subnet it's on, and get the segment_id, then boot using that | |
| 10:37:53 | gibi | rm_work: thanks ^^ | |
| 10:38:55 | rm_work | for the live-migrate case in nova, it's basically the same -- all the ports need to be pluggable to the new VM, so the new VM needs to have access to the same segments -- so it looks at all the ports on the old VM, gets the subnets->segments from them, and then does the migrate to a host that schedules with all of the relevant segments | |
| 10:39:38 | rm_work | you probably don't have to read the ENTIRETY of both specs to get the idea, and I don't know whether they ended up matching the implementation EXACTLY, but I can answer questions | |
| 10:40:47 | ileixe | rm_work: gibi: sorry to hijack, may I ask one more question? | |
| 10:40:54 | rm_work | o/ | |
| 10:41:05 | gibi | rm_work: thanks. This description helped me a lot | |
| 10:41:24 | ileixe | You said normal users were unaware of segments, does it mean normal user does not make VM using routed network? | |
| 10:41:41 | rm_work | when routed-networks are in use, the user is really not even aware of it | |
| 10:41:53 | rm_work | it's all handled inside nova/neutron | |
| 10:42:02 | rm_work | as far as the user is aware, they simply ask for a VM on a network, and it happens | |
| 10:42:03 | ileixe | Yes, so what happens to the normal user? | |
| 10:42:18 | ileixe | then they are not using routed network, right? | |
| 10:42:21 | gibi | rm_work: can we trade? :) I will support your way forward to either have the filter or the placement based sollution in ussuri, in reaturn you can help me reviewing the tempest patch? | |
| 10:42:34 | rm_work | so in the normal workflow, the user will do a nova boot, and it will go for scheduling, schedule to any HV, and then network plugging will be deferred to neutron instead of nova picking the IP | |
| 10:42:55 | rm_work | then neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side | |
| 10:44:04 | rm_work | routed networks is kinda just... something that a cloud either needs (and thus uses) or doesn't. in a cloud that's routed-network enabled, there really aren't *non-routed* networks, usually (at least, not that i've seen) | |
| 10:44:05 | gibi | so for first boot neutron will select the segment, but for live migrate nova should select a host that supports the same segment | |
| 10:44:10 | rm_work | it's really a network implementation choice | |
| 10:44:17 | rm_work | gibi: correct | |
| 10:44:31 | rm_work | currently, live-migrate cannot work in a routed-network enabled cloud | |
| 10:44:44 | gibi | rm_work: wooot! Finally I undestood it :) | |
| 10:45:04 | rm_work | because the scheduling doesn't understand it, and will pick some random HV based on other scheduling criteria, and then that HV may not actually be in the correct aggregate for plugging the old port :) | |
| 10:45:57 | gibi | plugging the old port in live migrate case fails becuase the port is already assigned to segment? while in the normal boot case the port is in a deferred state so the plug wont fail | |
| 10:46:04 | rm_work | yes | |
| 10:46:16 | rm_work | live-migrate needs to keep the old address (port) | |
| 10:46:25 | rm_work | a segment is a subnet attribute | |
| 10:46:33 | rm_work | port has an address (which is part of a subnet) | |
| 10:46:54 | sean-k-mooney | gibi: pluging the port on live migration would work if we live migrated to the same netwrok segment | |
| 10:46:56 | rm_work | so one Network can have many Subnets, each which is a separate segment | |
| 10:47:03 | gibi | ok, so during cold migrate if the port needs to be plugged to another segment then it will change IP address but it si not a problem as the guest is rebooted anyhow | |
| 10:47:12 | ileixe | "neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side" <- that's the point that I was unaware of | |
| 10:47:15 | rm_work | sean-k-mooney: right, so that's what we're talking about -- this simple filter allows for that | |
| 10:47:22 | ileixe | Thanks for explanation I will look at it | |
| 10:47:23 | sean-k-mooney | gibi: it should not change ip | |
| 10:47:57 | gibi | sean-k-mooney: but if it changes segment then it will change IP | |
| 10:48:04 | rm_work | I actually haven't even paid attention to cold-migrate, we don't do it either, or care :D it may have the same issue as live-migrate | |
| 10:48:14 | gibi | ahh I see | |
| 10:48:20 | rm_work | I am not sure | |
| 10:48:22 | sean-k-mooney | that my point its not vaild for the port to change its ip on a move operation so it cannot change segment | |
| 10:48:30 | rm_work | right | |
| 10:48:32 | sean-k-mooney | which is why it fails today | |
| 10:48:42 | rm_work | sean-k-mooney: https://review.opendev.org/#/c/709280/ | |
| 10:48:56 | gibi | rm_work: look, I understand you dont care but I as a nova maintainer needs to care. Hence my previous -1 as I did not understood the feature | |
| 10:49:00 | rm_work | this is a very very simple way to get live-migrate and other service-to-service use-cases unblocked | |
| 10:49:26 | sean-k-mooney | we use ip_allocation=defer to allow the ip to be allcoated when we bind the port but it cannoth change again after that point | |
| 10:49:36 | rm_work | gibi: right, sorry, I didn't mean that I don't care if all features work -- i mean i didn't care personally so hadn't checked whether it was the same issue -- this would solve it anyway if it is, and not affect it if it isn't | |
| 10:49:58 | rm_work | i'm not going to propose a feature that breaks other stuff :) | |
| 10:50:20 | gibi | sean-k-mooney: I rely on you about defining what is the expected behavior of a port during cold migrate in the current case so if you say it should not change IP then it accept that | |
| 10:50:42 | rm_work | sure, and in that case, this would allow to solve for cold-migrate as well | |
| 10:50:49 | rm_work | in the same way | |
| 10:51:18 | gibi | rm_work: no worries. I need to act as a guardian not you. I just explained why I was -1 | |
| 10:51:25 | sean-k-mooney | rm_work: just looking at it now but how is the segment id passed to nova? a schduler hint or somehting like that | |
| 10:51:34 | rm_work | sean-k-mooney: scheduler hint, yes | |
| 10:52:12 | sean-k-mooney | --hint routed_segments=96a9316a-54cb-4043-9ccc-b9cacd0d4d52 | |
| 10:52:25 | sean-k-mooney | so is that uuid a placement aggreate uuid | |
| 10:52:37 | sean-k-mooney | so we convert that to a member_of ? | |
| 10:52:41 | gibi | right now I like the simplicity of the filter, but I does not like matching the _name_ of the aggregate as that feels hackish | |
| 10:52:49 | rm_work | sean-k-mooney: this is a neutron segment_id | |
| 10:53:10 | rm_work | gibi: yeah I am not a huge fan of name-comparison as a matcher either, but per the spec it DOES work | |
| 10:53:19 | rm_work | so, this stuff is also in Placement as well | |
| 10:53:27 | rm_work | and in Placement it has a little tighter mapping | |
| 10:53:37 | sean-k-mooney | right so filters are not allowed to call other servcies rest apis so im wondering how nova knows if a host is connected to the segment | |
| 10:53:39 | gibi | rm_work: yeah I got it that this aggregate naming thing was how it is speced and implemented | |
| 10:53:46 | rm_work | sean-k-mooney: nova aggregates | |
| 10:53:55 | gibi | sean-k-mooney: ^^ | |
| 10:54:03 | sean-k-mooney | if we have nova host aggreates we dont need a new filter | |
| 10:54:13 | rm_work | sean-k-mooney: see for example http://paste.openstack.org/show/7YyK5FtL2OiXgErAjMbM/ | |
| 10:54:45 | rm_work | we don't know the correct nova aggregate | |
| 10:54:56 | rm_work | other services would know the segment_id | |
| 10:55:07 | rm_work | nove understands the host-aggregates internally and how they map to segments | |
| 10:55:43 | rm_work | sorry, when I say "we" in this case I mean other services | |
| 10:55:54 | rm_work | (I am speaking from the Octavia point of view, for example) | |
| 10:56:10 | rm_work | What Octavia has is a port, with a subnet and thus a segment_id | |
| 10:56:38 | rm_work | Octavia needs to tell Nova to schedule to the aggregate that has that segment_id | |
| 10:57:37 | rm_work | I guess you're saying that there exists a filter already that would allow us to specify an aggregate for scheduling, and we could just look that up? | |
| 10:58:03 | sean-k-mooney | i think there is one that would allow that yes | |
| 10:58:23 | sean-k-mooney | i tought we could use the json filter for that but it looks like the answer is no | |
| 10:58:59 | gibi | sean-k-mooney: AggregateInstanceExtraSpec filter works based on flavors and aggregate extra spec. but we only have the aggregate name to match | |
| 10:59:03 | rm_work | sean-k-mooney: so here is the example patch for how to make live-migrate work with this: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/ | |
| 10:59:37 | rm_work | gibi: i guess TECHNICALLY it's possible for other services to query nova for aggregate list, and do the matchup on their own | |
| 10:59:44 | sean-k-mooney | gibi: ya the aggreate image and extraspec filters are not the right granulatrity | |
| 10:59:53 | sean-k-mooney | i was considering the compute capabliteis filter | |
| 10:59:57 | rm_work | assuming we could pass an aggregate_id? | |
| 11:00:07 | sean-k-mooney | but setting the segment on each host would be a pain | |
| 11:00:20 | gibi | rm_work: you could not pass agg_id either, you can specify agg extra_spec in the flavor | |
| 11:00:31 | rm_work | in the *flavor*? | |
| 11:00:41 | rm_work | yeah, that's not workable | |
| 11:00:42 | sean-k-mooney | the issue with the filter based approch is if you change the default result set size form placment it can end up retruning only hosts that cant pass the filter | |
| 11:01:02 | rm_work | hmm | |
| 11:01:02 | sean-k-mooney | its the issue that cern had with only 10 results | |
| 11:01:04 | gibi | rm_work: yeah in the flavor, so I getting to grasp why you are in pain | |
| 11:01:15 | sean-k-mooney | /results/allocation_candiates | |
| 11:01:31 | rm_work | yeah we don't want to lock one flavor to one aggregate T_T | |
| 11:01:41 | rm_work | sean-k-mooney: hmm interesting... | |
| 11:02:05 | sean-k-mooney | i dont think we should do this as a filter. we could do it as a pre-filter | |
| 11:02:10 | gibi | sean-k-mooney: that is a valid point, but limit is configurable so the deployer can ask for a lot of candidates via a big limit to avoid this | |
| 11:02:16 | sean-k-mooney | and transfrom the hint into a member_of | |