Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-08
19:32:24 melwitt yeah, sorry, I meant rc2 vs wait until stein, that I was asking of you both
19:32:38 mriedem i'd say stein
19:32:43 dansmith for sure
19:32:47 melwitt k, cool
19:32:48 mriedem make sure there is no immediate regression,
19:32:49 mriedem then backports
19:32:53 sean-k-mooney mriedem: nova is pretty waterfall too in general that said the runways has made nova mor agile this cycle
19:33:09 mriedem hey man
19:33:10 melwitt so what about the placement perf thing? just try to land it for RC1 and not wait for RC2?
19:33:25 mriedem melwitt: my understanding on that is there are 2 fixes
19:33:41 mriedem cdent's is the more important of the two and is already approved
19:33:50 melwitt that's correct. one is approved, other is up for review
19:33:51 mriedem i.e. cdent's drops the perf by 50%
19:33:57 mriedem jay's drops that another 50%
19:34:14 sean-k-mooney dansmith: :) most of intel liked that word but followed water-scurm-fall developemnt instead
19:34:37 mriedem i'm pretty sure my first 10 years at ibm the dev model was really code-and-fix
19:34:56 mriedem the bestest of models
19:35:35 mriedem melwitt: i'll defer to efried and cdent and the placement boyz on how comfortable they are on jay's fix for rc2
19:35:39 sean-k-mooney mriedem: by drops the perf by 50% do you mean makes it better or worse
19:35:50 mriedem improves perf by 50%
19:35:58 sean-k-mooney :)
19:36:03 mriedem it's the dpdk of patches
19:36:27 sean-k-mooney mriedem: really fast and imposible to debug
19:36:49 mriedem heh that's a pretty good analogy for placement
19:37:05 melwitt mriedem: okay, sounds fair. efried and cdent, let me know what you think of it once you've reviewed
19:38:19 jaypipes melwitt: re: https://bugs.launchpad.net/nova/+bug/1746863, I thought we'd always said server groups were restricted to a single cell. is that not the case?
19:38:19 openstack Launchpad bug 1746863 in OpenStack Compute (nova) "scheduler affinity doesn't work with multiple cells" [High,In progress] - Assigned to melanie witt (melwitt)
19:38:31 cdent melwitt, mriedem : I'm happy to see them both go in
19:39:14 melwitt what is your assessment of the risk of the change somehow making the final release worse?
19:39:57 mriedem jaypipes: it's totally possible to have server group members wind up unintentionally in separate cells
19:40:04 mriedem even if in an affinity group
19:40:41 cdent melwitt: we talking about the placement thing on "final release worse"? the risk in those changes is very very low. the value is very very high.
19:41:33 melwitt jaypipes: they are, in a sense that affinity means same host (cells or not) and anti-affinity means different hosts. but the bug is that if you land on hostA for your first instance, because we don't look for members in all cells, we won't find that a group member is on hostA and therefore we need to co-locate instance2 to hostA for affinity if you want to add another host. I hope that makes sense
19:41:45 sean-k-mooney mriedem: jaypipes i guess mabe you should use dansmith's pre placement filter stuff to avoid that.
19:41:51 melwitt jaypipes: *if you want to add another instance
19:41:58 sean-k-mooney *could use
19:42:13 mriedem sean-k-mooney: how?
19:42:24 melwitt cdent: yeah, exactly. thanks for confirming it is low risk
19:42:24 mriedem placement doesn't know about server groups
19:42:32 mriedem nor cells
19:42:44 dansmith well, if we had a same-resource-provider thing we could kindof hack up a thing to do it via placement
19:43:02 dansmith but agree, it's not easy
19:43:43 dansmith it would be trivial to just fail a boot request for affinity if we can't talk to the cell where the other members are
19:43:53 sean-k-mooney mriedem: well the pre filter is for tenant affinity to a cell. i was thicnking if we had a request in a server group we could have a prefiltr that just picks a cell and only trys to place within that cell for the entire group
19:43:55 dansmith since we clearly can't honor the affinity goal
19:44:16 dansmith sean-k-mooney: that doesn't help us
19:44:23 dansmith sean-k-mooney: you might not be keeping tenants to cells
19:45:12 mriedem dansmith: yeah i think that's what gibi said on mel's patch
19:45:15 sean-k-mooney dansmith: i was not suggesting it was a depency just that if we detected there was an affinity group the only consider 1 cell for the request
19:45:20 dansmith mriedem: ack, haven't looked
19:45:37 sean-k-mooney dansmith: anyway it was just a tought.
19:45:49 mriedem we can still race our way around affinity and wind up in different cells
19:45:58 mriedem if you create the servers at the same time
19:46:24 mriedem and i think the late up-call doesn't help you there either
19:46:33 mriedem but it's been awhile since i've thought about this
19:46:38 sean-k-mooney ya ther isent really a way around that if you have multiple schdulers acitve.
19:46:40 dansmith we should be able to handle the race to multiple cells I think
19:47:22 dansmith if we're the first instance in a group, we schedule to any cell.. if we're not, we restrict to that cell, if we thought we were and then end up being the second one after a no-restriction thing, we reschedule
19:47:41 dansmith should be easy to check the mappings of other members to see if they're set to catch the last case I think
19:48:14 sean-k-mooney dansmith: would that not reqiure an up call from the compute to the top level cells db
19:48:19 dansmith no
19:48:26 dansmith it's all done in the scheduling superconductor loop
19:48:42 mriedem we don't reschedule within superconductor...
19:49:00 dansmith but we can
19:49:14 dansmith we don't currently because we shouldn't need to
19:49:36 dansmith scheduler re-queries if it can't claim resources, and conductor doesn't know if a compute will fail, so it can't reschedule at that point,
19:49:55 dansmith but if we have a reason, like we go create the instance in a cell, and then learn we lost the race, we can reschedule before we map that instance
19:50:51 dansmith you know,
19:52:10 dansmith I was going to spout off another potential way to solve the need for the affinity upcall in a similar way, but I need to think about that for a sec
19:52:29 dansmith but regardless for the cell race I think we can manage it in superconductor
19:54:51 sean-k-mooney dansmith: is there any reason we dont create placement aggregates for cells out of interest?
19:55:24 melwitt I hope somebody can write down these ideas somewhere so we don't forget in stein
19:55:41 sean-k-mooney if we did that consider a cell is just a member-of right?
19:56:14 sean-k-mooney all the info to create the aggregate is already in the cell mapping table
19:56:24 dansmith sean-k-mooney: I don't want to leak cells outside of nova in any way
19:56:44 dansmith sean-k-mooney: and I definitely don't want to auto-create aggregates in nova that align to cells
19:57:18 sean-k-mooney dansmith: aggregates in placement auto-created by nova you mean
19:57:54 dansmith sean-k-mooney: definitely not auto-created in nova, but probably not really in placement either
19:59:35 dansmith we can already filter hosts by cell in the scheduler
19:59:36 sean-k-mooney well a placement aggreate is just a group of resouce providers. there is no real inherint semantics beyond that. so you can model cells or regions or host aggregats equally
20:00:23 sean-k-mooney dansmith: true but if there was a agregate associated with placement could ignore everything not in that aggregate
20:00:44 sean-k-mooney anyway its just an idea.
20:00:55 openstackgerrit Merged openstack/nova master: Refactor cell_type in compute/api.py https://review.openstack.org/589833
20:21:07 melwitt mriedem: don't forget to please review these ocata quota fixes before you go on vacay https://review.openstack.org/588416 and the one above it
20:22:17 mriedem oh right
20:39:46 openstackgerrit melanie witt proposed openstack/nova master: Add a prelude release note for the 18.0.0 Rocky GA https://review.openstack.org/589303
20:51:13 openstackgerrit Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548
21:07:00 openstackgerrit Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548
21:13:30 openstackgerrit Jay Pipes proposed openstack/nova master: split gigantor SQL placement query into multiple https://review.openstack.org/590041
21:16:14 jaypipes mriedem, dansmith: is there a way to boot a Nova VM on behalf of another user/project?
21:16:30 dansmith I think that has been requested before
21:16:38 jaypipes mriedem: alternately, is there a way to transfer ownership of an instance once its been created?
21:16:50 dansmith also requested
21:16:54 jaypipes pretty sure that latter thing has been requested a few times, yeah
21:16:57 melwitt haha, Oath questions
21:17:00 dansmith yup
21:17:01 jaypipes :)
21:17:16 jaypipes melwitt: yup. trying to see if we can chuck the "ytenant" patches.
21:18:47 melwitt yeah, I kept talking about it for years "on behalf of" but there's not even support for that in keystone either, from what I understand. there's no such thing as a credential that records "I am A and I have obtained a token for B"
21:18:59 jaypipes mriedem, dansmith: BTW, https://review.openstack.org/#/c/590041 is all about that "debugging scheduler filters" thing...

Earlier   Later