Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-26
10:04:37 openstackgerrit Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494
10:16:28 openstackgerrit Lee Yarwood proposed openstack/nova master: virt: Provide block_device_info during rescue https://review.opendev.org/700811
10:16:29 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Add support for stable device rescue https://review.opendev.org/700812
10:16:29 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Report COMPUTE_RESCUE_BFV and check during rescue https://review.opendev.org/701429
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: api: Introduce microverion 2.82 allowing boot from volume rescue https://review.opendev.org/701430
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Extract _get_bdm_image_metadata into nova.utils https://review.opendev.org/705212
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Support boot from volume stable device instance rescue https://review.opendev.org/701431
10:18:18 gibi rm_work: responded in https://review.opendev.org/#/c/709280 I'm happy to help with technicalities but I missing the what the feature is supposed to achive and how the existing impl in neutron works
10:19:51 rm_work gibi: just read it, thanks!
10:20:07 rm_work so, the key is that it's not really a feature for end users at all -- it's for service integration cases
10:20:37 rm_work the octavia use-case is here: https://review.opendev.org/#/c/706153/
10:21:07 rm_work gibi: as it is, neutron has been creating aggregates for each segment_it in a routed network automatically, for like... 3 cycles already maybe?
10:24:12 rm_work so for example, if i take a look at my Staging environment's aggregate list, it looks like this: http://paste.openstack.org/show/7YyK5FtL2OiXgErAjMbM/
10:24:41 rm_work that aggregate was automatically created by neutron because it was configured as a routed network with a segment and those HVs were assigned to that segment in neutron
10:25:56 rm_work gibi: i don't think there's really any further work to be done -- this filter *just works* already for the intended purpose -- allowing services to hint to nova which aggregate to schedule to, based on network segment
10:26:05 rm_work the real challenge is just getting it merged :)
10:26:51 rm_work I know of at least two companies running this filter on live clouds already, one of them for many years
10:27:57 rm_work the second use-case is live-migrate, and I already have the basic patch for it (again, has been running for quite a while in some clouds) but I'm not spending the effort to get it all prettified yet until I have some confidence that this filter might be accepted
10:28:35 gibi rm_work: reading back ...
10:29:13 rm_work this is it though: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/
10:29:37 rm_work basically all I could ask from you is a +1 instead of a -1 if you understand what I'm going for here
10:30:26 rm_work trying to unlock useful features that have been stuck downstream because people didn't have the energy to do what I'm doing and post them upstream with tests and docs, and convince people to review :D
10:31:57 gibi rm_work: so the new scheduler hint routed_segments is a list of segment ids
10:32:03 rm_work yes
10:32:26 gibi rm_work: and the filter try to find an aggregate that has a name that ends with such segment id
10:32:28 rm_work users can't actually see segment_ids, so it's not really useful for the end-user side -- it's designed for service interactions
10:32:48 rm_work yes, as that's the neutron spec to auto-create aggregates with segment_ids with that name format
10:33:04 gibi rm_work: do you have a link to that neutron spec?
10:33:09 rm_work let me look
10:33:46 gibi so how the 'service' that calls nova knows the segment id it needs to pass in the scheduler hint?
10:36:21 rm_work nova side: https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html#proposed-change
10:36:43 rm_work neutron side: https://specs.openstack.org/openstack/neutron-specs/specs/newton/routed-networks.html
10:36:50 gibi thanks
10:37:17 gibi I have to read up
10:37:21 rm_work gibi: so for the octavia case, the issue is that we need both of the service-vms we build to be in the same segment as the VIP (an unbound neutron port) we create
10:37:37 rm_work so, we simply look at that port, then look at the subnet it's on, and get the segment_id, then boot using that
10:37:53 gibi rm_work: thanks ^^
10:38:55 rm_work for the live-migrate case in nova, it's basically the same -- all the ports need to be pluggable to the new VM, so the new VM needs to have access to the same segments -- so it looks at all the ports on the old VM, gets the subnets->segments from them, and then does the migrate to a host that schedules with all of the relevant segments
10:39:38 rm_work you probably don't have to read the ENTIRETY of both specs to get the idea, and I don't know whether they ended up matching the implementation EXACTLY, but I can answer questions
10:40:47 ileixe rm_work: gibi: sorry to hijack, may I ask one more question?
10:40:54 rm_work o/
10:41:05 gibi rm_work: thanks. This description helped me a lot
10:41:24 ileixe You said normal users were unaware of segments, does it mean normal user does not make VM using routed network?
10:41:41 rm_work when routed-networks are in use, the user is really not even aware of it
10:41:53 rm_work it's all handled inside nova/neutron
10:42:02 rm_work as far as the user is aware, they simply ask for a VM on a network, and it happens
10:42:03 ileixe Yes, so what happens to the normal user?
10:42:18 ileixe then they are not using routed network, right?
10:42:21 gibi rm_work: can we trade? :) I will support your way forward to either have the filter or the placement based sollution in ussuri, in reaturn you can help me reviewing the tempest patch?
10:42:34 rm_work so in the normal workflow, the user will do a nova boot, and it will go for scheduling, schedule to any HV, and then network plugging will be deferred to neutron instead of nova picking the IP
10:42:55 rm_work then neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side
10:44:04 rm_work routed networks is kinda just... something that a cloud either needs (and thus uses) or doesn't. in a cloud that's routed-network enabled, there really aren't *non-routed* networks, usually (at least, not that i've seen)
10:44:05 gibi so for first boot neutron will select the segment, but for live migrate nova should select a host that supports the same segment
10:44:10 rm_work it's really a network implementation choice
10:44:17 rm_work gibi: correct
10:44:31 rm_work currently, live-migrate cannot work in a routed-network enabled cloud
10:44:44 gibi rm_work: wooot! Finally I undestood it :)
10:45:04 rm_work because the scheduling doesn't understand it, and will pick some random HV based on other scheduling criteria, and then that HV may not actually be in the correct aggregate for plugging the old port :)
10:45:57 gibi plugging the old port in live migrate case fails becuase the port is already assigned to segment? while in the normal boot case the port is in a deferred state so the plug wont fail
10:46:04 rm_work yes
10:46:16 rm_work live-migrate needs to keep the old address (port)
10:46:25 rm_work a segment is a subnet attribute
10:46:33 rm_work port has an address (which is part of a subnet)
10:46:54 sean-k-mooney gibi: pluging the port on live migration would work if we live migrated to the same netwrok segment
10:46:56 rm_work so one Network can have many Subnets, each which is a separate segment
10:47:03 gibi ok, so during cold migrate if the port needs to be plugged to another segment then it will change IP address but it si not a problem as the guest is rebooted anyhow
10:47:12 ileixe "neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side" <- that's the point that I was unaware of
10:47:15 rm_work sean-k-mooney: right, so that's what we're talking about -- this simple filter allows for that
10:47:22 ileixe Thanks for explanation I will look at it
10:47:23 sean-k-mooney gibi: it should not change ip
10:47:57 gibi sean-k-mooney: but if it changes segment then it will change IP
10:48:04 rm_work I actually haven't even paid attention to cold-migrate, we don't do it either, or care :D it may have the same issue as live-migrate
10:48:14 gibi ahh I see
10:48:20 rm_work I am not sure
10:48:22 sean-k-mooney that my point its not vaild for the port to change its ip on a move operation so it cannot change segment
10:48:30 rm_work right
10:48:32 sean-k-mooney which is why it fails today
10:48:42 rm_work sean-k-mooney: https://review.opendev.org/#/c/709280/
10:48:56 gibi rm_work: look, I understand you dont care but I as a nova maintainer needs to care. Hence my previous -1 as I did not understood the feature
10:49:00 rm_work this is a very very simple way to get live-migrate and other service-to-service use-cases unblocked
10:49:26 sean-k-mooney we use ip_allocation=defer to allow the ip to be allcoated when we bind the port but it cannoth change again after that point
10:49:36 rm_work gibi: right, sorry, I didn't mean that I don't care if all features work -- i mean i didn't care personally so hadn't checked whether it was the same issue -- this would solve it anyway if it is, and not affect it if it isn't
10:49:58 rm_work i'm not going to propose a feature that breaks other stuff :)
10:50:20 gibi sean-k-mooney: I rely on you about defining what is the expected behavior of a port during cold migrate in the current case so if you say it should not change IP then it accept that
10:50:42 rm_work sure, and in that case, this would allow to solve for cold-migrate as well
10:50:49 rm_work in the same way
10:51:18 gibi rm_work: no worries. I need to act as a guardian not you. I just explained why I was -1
10:51:25 sean-k-mooney rm_work: just looking at it now but how is the segment id passed to nova? a schduler hint or somehting like that
10:51:34 rm_work sean-k-mooney: scheduler hint, yes
10:52:12 sean-k-mooney --hint routed_segments=96a9316a-54cb-4043-9ccc-b9cacd0d4d52
10:52:25 sean-k-mooney so is that uuid a placement aggreate uuid
10:52:37 sean-k-mooney so we convert that to a member_of ?
10:52:41 gibi right now I like the simplicity of the filter, but I does not like matching the _name_ of the aggregate as that feels hackish
10:52:49 rm_work sean-k-mooney: this is a neutron segment_id
10:53:10 rm_work gibi: yeah I am not a huge fan of name-comparison as a matcher either, but per the spec it DOES work
10:53:19 rm_work so, this stuff is also in Placement as well
10:53:27 rm_work and in Placement it has a little tighter mapping
10:53:37 sean-k-mooney right so filters are not allowed to call other servcies rest apis so im wondering how nova knows if a host is connected to the segment
10:53:39 gibi rm_work: yeah I got it that this aggregate naming thing was how it is speced and implemented
10:53:46 rm_work sean-k-mooney: nova aggregates

Earlier   Later