Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-25
20:11:30 prometheanfire sgtm
20:15:55 dansmith efried: recheck to make sure the change works on the old/current version too you mean?
20:16:00 dansmith but yeah, I guess that's the process
20:33:07 sean-k-mooney am i right in thinkin that https://review.opendev.org/#/c/699554/17 or actully the patch before it are technically all that are need to boot a vm with a cyborg deivce
20:33:59 sean-k-mooney the rpc bump patch is in merge conflict so i cant currently test my multinode job with that so i was temporaly going to depend on the earlier patch to get a ci run tonight
20:35:24 sean-k-mooney the patches after https://review.opendev.org/#/c/699554/17 while important are reall just nice to haves in terms of actully executing tempest full to check the job runs correctly
20:38:45 sean-k-mooney ill assume yes and kick off a run. ill be away for an hour or so and check in with it later o/
20:38:47 openstackgerrit Merged openstack/nova stable/train: Create instance action when burying in cell0 https://review.opendev.org/701279
20:46:58 efried dansmith: recheck because currently it is running against nova that doesn't have this patch
20:52:16 openstackgerrit Jason Anderson proposed openstack/nova master: Update resource tracker to use per-hypervisor lock https://review.opendev.org/709832
21:41:10 openstackgerrit Eric Fried proposed openstack/nova master: Name Enums https://review.opendev.org/708707
#openstack-nova - 2020-02-26
00:48:54 sean-k-mooney dansmith: efried just heading to bed but my multinode cyborg tempest full job https://review.opendev.org/#/c/709641/5 just completed https://48ef08cde8cc22034a1d-8011a2266d21f0c09baf1c83d6d5002e.ssl.cf5.rackcdn.com/709641/5/check/cyborg-multinode-tempest-full/e4d260f/testr_results.html
00:49:40 sean-k-mooney i have only skimmed them quickly but i belive it shows that vms can be booted with cyborg flavor initail since tempest.api.compute.servers.test_create_server.ServersTestJSON.test_host_name_is_same_as_server_name passed i change the default flavor to request a fake deivce
00:50:21 sean-k-mooney it also repoduced teh 401 issue where the keystone middelware eventually starts rejecting the token because it think it expires
00:51:19 sean-k-mooney and i can see the live migration operations fail with Details: {'code': 400, 'message': 'No valid host was found. Unable to move instance 229ab495-b151-4522-b8dd-fa5818e302dd to host ubuntu-bionic-vexxhost-sjc1-0014809343. The instance has complex allocations on the source host so move cannot be forced.'}
00:52:24 sean-k-mooney so i think that is implying that 1 the vm was created with the cybrog resouces and two live migration is being rejected before we get to the driver which is good
00:52:45 sean-k-mooney that was from test_live_block_migration
00:54:45 sean-k-mooney anyway im calling it a night o/ hopefully that will be useful as it is repoducing the same issues i was seeing and its now automated so easy to rerun
01:55:13 openstackgerrit Brin Zhang proposed openstack/nova master: Add test coverage of existing os-volumes-attachments policies https://review.opendev.org/709929
02:00:51 openstackgerrit Brin Zhang proposed openstack/nova master: Introduce scope_types in os-volumes-attachments policy https://review.opendev.org/709388
06:51:48 openstackgerrit Brin Zhang proposed openstack/nova master: Fix os-volumes-attachments policy to be admin_or_owner https://review.opendev.org/709955
07:35:32 openstack bug 1864776 in OpenStack Compute (nova) "os-volumes-attachments API policy is allowed for everyone even policy defaults is admin_or_owner" [Undecided,In progress] https://launchpad.net/bugs/1864776 - Assigned to Brin Zhang (zhangbailin)
07:35:32 brinzhang melwitt, gmann, stephenfin: is bug 1864776 real? can you all check?
07:40:08 brinzhang it's all of the author contexts.}
07:40:08 brinzhang gmann: the new policy of admin_api is {self.legacy_admin_context, self.system_admin_context, self.project_admin_context}, and new policy of the admin_or_owner is {self.legacy_admin_context, self.system_admin_context, self.project_admin_context, self.project_member_context, self.system_member_context, self.system_reader_context, self.system_foo_context, self.project_foo_context, self.project_reader_context, self.other_project_member_context},
07:40:29 brinzhang gmann: Understand of this right?
10:04:37 openstackgerrit Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494
10:16:28 openstackgerrit Lee Yarwood proposed openstack/nova master: virt: Provide block_device_info during rescue https://review.opendev.org/700811
10:16:29 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Report COMPUTE_RESCUE_BFV and check during rescue https://review.opendev.org/701429
10:16:29 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Add support for stable device rescue https://review.opendev.org/700812
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: libvirt: Support boot from volume stable device instance rescue https://review.opendev.org/701431
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: compute: Extract _get_bdm_image_metadata into nova.utils https://review.opendev.org/705212
10:16:30 openstackgerrit Lee Yarwood proposed openstack/nova master: api: Introduce microverion 2.82 allowing boot from volume rescue https://review.opendev.org/701430
10:18:18 gibi rm_work: responded in https://review.opendev.org/#/c/709280 I'm happy to help with technicalities but I missing the what the feature is supposed to achive and how the existing impl in neutron works
10:19:51 rm_work gibi: just read it, thanks!
10:20:07 rm_work so, the key is that it's not really a feature for end users at all -- it's for service integration cases
10:20:37 rm_work the octavia use-case is here: https://review.opendev.org/#/c/706153/
10:21:07 rm_work gibi: as it is, neutron has been creating aggregates for each segment_it in a routed network automatically, for like... 3 cycles already maybe?
10:24:12 rm_work so for example, if i take a look at my Staging environment's aggregate list, it looks like this: http://paste.openstack.org/show/7YyK5FtL2OiXgErAjMbM/
10:24:41 rm_work that aggregate was automatically created by neutron because it was configured as a routed network with a segment and those HVs were assigned to that segment in neutron
10:25:56 rm_work gibi: i don't think there's really any further work to be done -- this filter *just works* already for the intended purpose -- allowing services to hint to nova which aggregate to schedule to, based on network segment
10:26:05 rm_work the real challenge is just getting it merged :)
10:26:51 rm_work I know of at least two companies running this filter on live clouds already, one of them for many years
10:27:57 rm_work the second use-case is live-migrate, and I already have the basic patch for it (again, has been running for quite a while in some clouds) but I'm not spending the effort to get it all prettified yet until I have some confidence that this filter might be accepted
10:28:35 gibi rm_work: reading back ...
10:29:13 rm_work this is it though: http://paste.openstack.org/show/Z8ZYksr4uisFUddFoyOu/
10:29:37 rm_work basically all I could ask from you is a +1 instead of a -1 if you understand what I'm going for here
10:30:26 rm_work trying to unlock useful features that have been stuck downstream because people didn't have the energy to do what I'm doing and post them upstream with tests and docs, and convince people to review :D
10:31:57 gibi rm_work: so the new scheduler hint routed_segments is a list of segment ids
10:32:03 rm_work yes
10:32:26 gibi rm_work: and the filter try to find an aggregate that has a name that ends with such segment id
10:32:28 rm_work users can't actually see segment_ids, so it's not really useful for the end-user side -- it's designed for service interactions
10:32:48 rm_work yes, as that's the neutron spec to auto-create aggregates with segment_ids with that name format
10:33:04 gibi rm_work: do you have a link to that neutron spec?
10:33:09 rm_work let me look
10:33:46 gibi so how the 'service' that calls nova knows the segment id it needs to pass in the scheduler hint?
10:36:21 rm_work nova side: https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html#proposed-change
10:36:43 rm_work neutron side: https://specs.openstack.org/openstack/neutron-specs/specs/newton/routed-networks.html
10:36:50 gibi thanks
10:37:17 gibi I have to read up
10:37:21 rm_work gibi: so for the octavia case, the issue is that we need both of the service-vms we build to be in the same segment as the VIP (an unbound neutron port) we create
10:37:37 rm_work so, we simply look at that port, then look at the subnet it's on, and get the segment_id, then boot using that
10:37:53 gibi rm_work: thanks ^^
10:38:55 rm_work for the live-migrate case in nova, it's basically the same -- all the ports need to be pluggable to the new VM, so the new VM needs to have access to the same segments -- so it looks at all the ports on the old VM, gets the subnets->segments from them, and then does the migrate to a host that schedules with all of the relevant segments
10:39:38 rm_work you probably don't have to read the ENTIRETY of both specs to get the idea, and I don't know whether they ended up matching the implementation EXACTLY, but I can answer questions
10:40:47 ileixe rm_work: gibi: sorry to hijack, may I ask one more question?
10:40:54 rm_work o/
10:41:05 gibi rm_work: thanks. This description helped me a lot
10:41:24 ileixe You said normal users were unaware of segments, does it mean normal user does not make VM using routed network?
10:41:41 rm_work when routed-networks are in use, the user is really not even aware of it
10:41:53 rm_work it's all handled inside nova/neutron
10:42:02 rm_work as far as the user is aware, they simply ask for a VM on a network, and it happens
10:42:03 ileixe Yes, so what happens to the normal user?
10:42:18 ileixe then they are not using routed network, right?
10:42:21 gibi rm_work: can we trade? :) I will support your way forward to either have the filter or the placement based sollution in ussuri, in reaturn you can help me reviewing the tempest patch?
10:42:34 rm_work so in the normal workflow, the user will do a nova boot, and it will go for scheduling, schedule to any HV, and then network plugging will be deferred to neutron instead of nova picking the IP
10:42:55 rm_work then neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side
10:44:04 rm_work routed networks is kinda just... something that a cloud either needs (and thus uses) or doesn't. in a cloud that's routed-network enabled, there really aren't *non-routed* networks, usually (at least, not that i've seen)
10:44:05 gibi so for first boot neutron will select the segment, but for live migrate nova should select a host that supports the same segment
10:44:10 rm_work it's really a network implementation choice
10:44:17 rm_work gibi: correct
10:44:31 rm_work currently, live-migrate cannot work in a routed-network enabled cloud
10:44:44 gibi rm_work: wooot! Finally I undestood it :)
10:45:04 rm_work because the scheduling doesn't understand it, and will pick some random HV based on other scheduling criteria, and then that HV may not actually be in the correct aggregate for plugging the old port :)
10:45:57 gibi plugging the old port in live migrate case fails becuase the port is already assigned to segment? while in the normal boot case the port is in a deferred state so the plug wont fail
10:46:04 rm_work yes
10:46:16 rm_work live-migrate needs to keep the old address (port)
10:46:25 rm_work a segment is a subnet attribute
10:46:33 rm_work port has an address (which is part of a subnet)
10:46:54 sean-k-mooney gibi: pluging the port on live migration would work if we live migrated to the same netwrok segment
10:46:56 rm_work so one Network can have many Subnets, each which is a separate segment
10:47:03 gibi ok, so during cold migrate if the port needs to be plugged to another segment then it will change IP address but it si not a problem as the guest is rebooted anyhow
10:47:12 ileixe "neutron will look at the HV it was scheduled to, pick an appropriate segment, and do the IP assignment/plug on the neutron side" <- that's the point that I was unaware of
10:47:15 rm_work sean-k-mooney: right, so that's what we're talking about -- this simple filter allows for that
10:47:22 ileixe Thanks for explanation I will look at it
10:47:23 sean-k-mooney gibi: it should not change ip
10:47:57 gibi sean-k-mooney: but if it changes segment then it will change IP
10:48:04 rm_work I actually haven't even paid attention to cold-migrate, we don't do it either, or care :D it may have the same issue as live-migrate
10:48:14 gibi ahh I see
10:48:20 rm_work I am not sure

Earlier   Later