Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-08
19:35:39 sean-k-mooney mriedem: by drops the perf by 50% do you mean makes it better or worse
19:35:50 mriedem improves perf by 50%
19:35:58 sean-k-mooney :)
19:36:03 mriedem it's the dpdk of patches
19:36:27 sean-k-mooney mriedem: really fast and imposible to debug
19:36:49 mriedem heh that's a pretty good analogy for placement
19:37:05 melwitt mriedem: okay, sounds fair. efried and cdent, let me know what you think of it once you've reviewed
19:38:19 openstack Launchpad bug 1746863 in OpenStack Compute (nova) "scheduler affinity doesn't work with multiple cells" [High,In progress] - Assigned to melanie witt (melwitt)
19:38:19 jaypipes melwitt: re: https://bugs.launchpad.net/nova/+bug/1746863, I thought we'd always said server groups were restricted to a single cell. is that not the case?
19:38:31 cdent melwitt, mriedem : I'm happy to see them both go in
19:39:14 melwitt what is your assessment of the risk of the change somehow making the final release worse?
19:39:57 mriedem jaypipes: it's totally possible to have server group members wind up unintentionally in separate cells
19:40:04 mriedem even if in an affinity group
19:40:41 cdent melwitt: we talking about the placement thing on "final release worse"? the risk in those changes is very very low. the value is very very high.
19:41:33 melwitt jaypipes: they are, in a sense that affinity means same host (cells or not) and anti-affinity means different hosts. but the bug is that if you land on hostA for your first instance, because we don't look for members in all cells, we won't find that a group member is on hostA and therefore we need to co-locate instance2 to hostA for affinity if you want to add another host. I hope that makes sense
19:41:45 sean-k-mooney mriedem: jaypipes i guess mabe you should use dansmith's pre placement filter stuff to avoid that.
19:41:51 melwitt jaypipes: *if you want to add another instance
19:41:58 sean-k-mooney *could use
19:42:13 mriedem sean-k-mooney: how?
19:42:24 mriedem placement doesn't know about server groups
19:42:24 melwitt cdent: yeah, exactly. thanks for confirming it is low risk
19:42:32 mriedem nor cells
19:42:44 dansmith well, if we had a same-resource-provider thing we could kindof hack up a thing to do it via placement
19:43:02 dansmith but agree, it's not easy
19:43:43 dansmith it would be trivial to just fail a boot request for affinity if we can't talk to the cell where the other members are
19:43:53 sean-k-mooney mriedem: well the pre filter is for tenant affinity to a cell. i was thicnking if we had a request in a server group we could have a prefiltr that just picks a cell and only trys to place within that cell for the entire group
19:43:55 dansmith since we clearly can't honor the affinity goal
19:44:16 dansmith sean-k-mooney: that doesn't help us
19:44:23 dansmith sean-k-mooney: you might not be keeping tenants to cells
19:45:12 mriedem dansmith: yeah i think that's what gibi said on mel's patch
19:45:15 sean-k-mooney dansmith: i was not suggesting it was a depency just that if we detected there was an affinity group the only consider 1 cell for the request
19:45:20 dansmith mriedem: ack, haven't looked
19:45:37 sean-k-mooney dansmith: anyway it was just a tought.
19:45:49 mriedem we can still race our way around affinity and wind up in different cells
19:45:58 mriedem if you create the servers at the same time
19:46:24 mriedem and i think the late up-call doesn't help you there either
19:46:33 mriedem but it's been awhile since i've thought about this
19:46:38 sean-k-mooney ya ther isent really a way around that if you have multiple schdulers acitve.
19:46:40 dansmith we should be able to handle the race to multiple cells I think
19:47:22 dansmith if we're the first instance in a group, we schedule to any cell.. if we're not, we restrict to that cell, if we thought we were and then end up being the second one after a no-restriction thing, we reschedule
19:47:41 dansmith should be easy to check the mappings of other members to see if they're set to catch the last case I think
19:48:14 sean-k-mooney dansmith: would that not reqiure an up call from the compute to the top level cells db
19:48:19 dansmith no
19:48:26 dansmith it's all done in the scheduling superconductor loop
19:48:42 mriedem we don't reschedule within superconductor...
19:49:00 dansmith but we can
19:49:14 dansmith we don't currently because we shouldn't need to
19:49:36 dansmith scheduler re-queries if it can't claim resources, and conductor doesn't know if a compute will fail, so it can't reschedule at that point,
19:49:55 dansmith but if we have a reason, like we go create the instance in a cell, and then learn we lost the race, we can reschedule before we map that instance
19:50:51 dansmith you know,
19:52:10 dansmith I was going to spout off another potential way to solve the need for the affinity upcall in a similar way, but I need to think about that for a sec
19:52:29 dansmith but regardless for the cell race I think we can manage it in superconductor
19:54:51 sean-k-mooney dansmith: is there any reason we dont create placement aggregates for cells out of interest?
19:55:24 melwitt I hope somebody can write down these ideas somewhere so we don't forget in stein
19:55:41 sean-k-mooney if we did that consider a cell is just a member-of right?
19:56:14 sean-k-mooney all the info to create the aggregate is already in the cell mapping table
19:56:24 dansmith sean-k-mooney: I don't want to leak cells outside of nova in any way
19:56:44 dansmith sean-k-mooney: and I definitely don't want to auto-create aggregates in nova that align to cells
19:57:18 sean-k-mooney dansmith: aggregates in placement auto-created by nova you mean
19:57:54 dansmith sean-k-mooney: definitely not auto-created in nova, but probably not really in placement either
19:59:35 dansmith we can already filter hosts by cell in the scheduler
19:59:36 sean-k-mooney well a placement aggreate is just a group of resouce providers. there is no real inherint semantics beyond that. so you can model cells or regions or host aggregats equally
20:00:23 sean-k-mooney dansmith: true but if there was a agregate associated with placement could ignore everything not in that aggregate
20:00:44 sean-k-mooney anyway its just an idea.
20:00:55 openstackgerrit Merged openstack/nova master: Refactor cell_type in compute/api.py https://review.openstack.org/589833
20:21:07 melwitt mriedem: don't forget to please review these ocata quota fixes before you go on vacay https://review.openstack.org/588416 and the one above it
20:22:17 mriedem oh right
20:39:46 openstackgerrit melanie witt proposed openstack/nova master: Add a prelude release note for the 18.0.0 Rocky GA https://review.openstack.org/589303
20:51:13 openstackgerrit Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548
21:07:00 openstackgerrit Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548
21:13:30 openstackgerrit Jay Pipes proposed openstack/nova master: split gigantor SQL placement query into multiple https://review.openstack.org/590041
21:16:14 jaypipes mriedem, dansmith: is there a way to boot a Nova VM on behalf of another user/project?
21:16:30 dansmith I think that has been requested before
21:16:38 jaypipes mriedem: alternately, is there a way to transfer ownership of an instance once its been created?
21:16:50 dansmith also requested
21:16:54 jaypipes pretty sure that latter thing has been requested a few times, yeah
21:16:57 melwitt haha, Oath questions
21:17:00 dansmith yup
21:17:01 jaypipes :)
21:17:16 jaypipes melwitt: yup. trying to see if we can chuck the "ytenant" patches.
21:18:47 melwitt yeah, I kept talking about it for years "on behalf of" but there's not even support for that in keystone either, from what I understand. there's no such thing as a credential that records "I am A and I have obtained a token for B"
21:18:59 jaypipes mriedem, dansmith: BTW, https://review.openstack.org/#/c/590041 is all about that "debugging scheduler filters" thing...
21:19:22 jaypipes mriedem, dansmith: or rather the "NoValidHosts oh noes!" thing.
21:19:34 dansmith both would be complicated, especially around image, flavor, and affinity things that would be violated by the new tenant
21:19:34 jaypipes bnemec-pto: https://review.openstack.org/#/c/590041
21:19:38 openstackgerrit Merged openstack/nova master: xx_instance_type_id in list_migrations should be integer https://review.openstack.org/588481
21:20:08 melwitt yeah, transfer of ownership would be very difficult
21:23:18 melwitt but for "on behalf of", an idea I had was if we were to take a --project-id, --user-id parameter to nova boot, that you could pass if you're admin, then we create the instance with the given project/user instead of the admin's. and we'd have to record in the logs that it happened, "Project/user admin/admin has requested instance create for person/person" so it's traceable in some way
21:24:08 dansmith well,
21:24:18 dansmith presumably you get admin's access to images and flavors?
21:24:28 dansmith so what do we do if the user tries to rebuild?
21:24:31 dansmith or resize?
21:24:47 dansmith and do you get admin's affinity or the user's?
21:24:52 melwitt the project/user recorded in the database will be theirs (the user's), not the admins
21:24:54 dansmith what about when they rebuild/resize/etc?
21:24:59 melwitt after the admin creates it, no more admin involved
21:25:34 dansmith which means they can't rebuild if the admin built with an image they can't access yeah?
21:25:38 jaypipes dansmith: it's a "let's have the admin hand-hold you for a little while, then after that, you can use it" sort of anti-feature.
21:25:46 melwitt dansmith: yeah
21:25:52 dansmith jaypipes: yes, anti-feature is a good name for it :)

Earlier   Later