| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-08 | |||
| 19:31:00 | mriedem | "ok so when do we do rc2 and rc3?" | |
| 19:31:09 | mriedem | "we don't, get your fixes into rc1 or don't cut rc1" | |
| 19:31:12 | mriedem | "BUT!!!" | |
| 19:31:25 | dansmith | well, we did have those couple of cycles where we always had an rc2 for translations on the things we got into rc1 | |
| 19:31:44 | mriedem | right, but the PMs at ibm planned for multiple candidates because they knew we had a backlog of bugs | |
| 19:31:48 | mriedem | total waterfall | |
| 19:31:53 | dansmith | heh yeah, and, | |
| 19:32:01 | dansmith | why call it rc1 if there's no rc2 amirite/ | |
| 19:32:10 | mriedem | that's deep | |
| 19:32:21 | dansmith | it's like an outline with one bullet.. no larger sin. | |
| 19:32:24 | melwitt | yeah, sorry, I meant rc2 vs wait until stein, that I was asking of you both | |
| 19:32:38 | mriedem | i'd say stein | |
| 19:32:43 | dansmith | for sure | |
| 19:32:47 | melwitt | k, cool | |
| 19:32:48 | mriedem | make sure there is no immediate regression, | |
| 19:32:49 | mriedem | then backports | |
| 19:32:53 | sean-k-mooney | mriedem: nova is pretty waterfall too in general that said the runways has made nova mor agile this cycle | |
| 19:33:09 | mriedem | hey man | |
| 19:33:10 | melwitt | so what about the placement perf thing? just try to land it for RC1 and not wait for RC2? | |
| 19:33:25 | mriedem | melwitt: my understanding on that is there are 2 fixes | |
| 19:33:41 | mriedem | cdent's is the more important of the two and is already approved | |
| 19:33:50 | melwitt | that's correct. one is approved, other is up for review | |
| 19:33:51 | mriedem | i.e. cdent's drops the perf by 50% | |
| 19:33:57 | mriedem | jay's drops that another 50% | |
| 19:34:14 | sean-k-mooney | dansmith: :) most of intel liked that word but followed water-scurm-fall developemnt instead | |
| 19:34:37 | mriedem | i'm pretty sure my first 10 years at ibm the dev model was really code-and-fix | |
| 19:34:56 | mriedem | the bestest of models | |
| 19:35:35 | mriedem | melwitt: i'll defer to efried and cdent and the placement boyz on how comfortable they are on jay's fix for rc2 | |
| 19:35:39 | sean-k-mooney | mriedem: by drops the perf by 50% do you mean makes it better or worse | |
| 19:35:50 | mriedem | improves perf by 50% | |
| 19:35:58 | sean-k-mooney | :) | |
| 19:36:03 | mriedem | it's the dpdk of patches | |
| 19:36:27 | sean-k-mooney | mriedem: really fast and imposible to debug | |
| 19:36:49 | mriedem | heh that's a pretty good analogy for placement | |
| 19:37:05 | melwitt | mriedem: okay, sounds fair. efried and cdent, let me know what you think of it once you've reviewed | |
| 19:38:19 | jaypipes | melwitt: re: https://bugs.launchpad.net/nova/+bug/1746863, I thought we'd always said server groups were restricted to a single cell. is that not the case? | |
| 19:38:19 | openstack | Launchpad bug 1746863 in OpenStack Compute (nova) "scheduler affinity doesn't work with multiple cells" [High,In progress] - Assigned to melanie witt (melwitt) | |
| 19:38:31 | cdent | melwitt, mriedem : I'm happy to see them both go in | |
| 19:39:14 | melwitt | what is your assessment of the risk of the change somehow making the final release worse? | |
| 19:39:57 | mriedem | jaypipes: it's totally possible to have server group members wind up unintentionally in separate cells | |
| 19:40:04 | mriedem | even if in an affinity group | |
| 19:40:41 | cdent | melwitt: we talking about the placement thing on "final release worse"? the risk in those changes is very very low. the value is very very high. | |
| 19:41:33 | melwitt | jaypipes: they are, in a sense that affinity means same host (cells or not) and anti-affinity means different hosts. but the bug is that if you land on hostA for your first instance, because we don't look for members in all cells, we won't find that a group member is on hostA and therefore we need to co-locate instance2 to hostA for affinity if you want to add another host. I hope that makes sense | |
| 19:41:45 | sean-k-mooney | mriedem: jaypipes i guess mabe you should use dansmith's pre placement filter stuff to avoid that. | |
| 19:41:51 | melwitt | jaypipes: *if you want to add another instance | |
| 19:41:58 | sean-k-mooney | *could use | |
| 19:42:13 | mriedem | sean-k-mooney: how? | |
| 19:42:24 | melwitt | cdent: yeah, exactly. thanks for confirming it is low risk | |
| 19:42:24 | mriedem | placement doesn't know about server groups | |
| 19:42:32 | mriedem | nor cells | |
| 19:42:44 | dansmith | well, if we had a same-resource-provider thing we could kindof hack up a thing to do it via placement | |
| 19:43:02 | dansmith | but agree, it's not easy | |
| 19:43:43 | dansmith | it would be trivial to just fail a boot request for affinity if we can't talk to the cell where the other members are | |
| 19:43:53 | sean-k-mooney | mriedem: well the pre filter is for tenant affinity to a cell. i was thicnking if we had a request in a server group we could have a prefiltr that just picks a cell and only trys to place within that cell for the entire group | |
| 19:43:55 | dansmith | since we clearly can't honor the affinity goal | |
| 19:44:16 | dansmith | sean-k-mooney: that doesn't help us | |
| 19:44:23 | dansmith | sean-k-mooney: you might not be keeping tenants to cells | |
| 19:45:12 | mriedem | dansmith: yeah i think that's what gibi said on mel's patch | |
| 19:45:15 | sean-k-mooney | dansmith: i was not suggesting it was a depency just that if we detected there was an affinity group the only consider 1 cell for the request | |
| 19:45:20 | dansmith | mriedem: ack, haven't looked | |
| 19:45:37 | sean-k-mooney | dansmith: anyway it was just a tought. | |
| 19:45:49 | mriedem | we can still race our way around affinity and wind up in different cells | |
| 19:45:58 | mriedem | if you create the servers at the same time | |
| 19:46:24 | mriedem | and i think the late up-call doesn't help you there either | |
| 19:46:33 | mriedem | but it's been awhile since i've thought about this | |
| 19:46:38 | sean-k-mooney | ya ther isent really a way around that if you have multiple schdulers acitve. | |
| 19:46:40 | dansmith | we should be able to handle the race to multiple cells I think | |
| 19:47:22 | dansmith | if we're the first instance in a group, we schedule to any cell.. if we're not, we restrict to that cell, if we thought we were and then end up being the second one after a no-restriction thing, we reschedule | |
| 19:47:41 | dansmith | should be easy to check the mappings of other members to see if they're set to catch the last case I think | |
| 19:48:14 | sean-k-mooney | dansmith: would that not reqiure an up call from the compute to the top level cells db | |
| 19:48:19 | dansmith | no | |
| 19:48:26 | dansmith | it's all done in the scheduling superconductor loop | |
| 19:48:42 | mriedem | we don't reschedule within superconductor... | |
| 19:49:00 | dansmith | but we can | |
| 19:49:14 | dansmith | we don't currently because we shouldn't need to | |
| 19:49:36 | dansmith | scheduler re-queries if it can't claim resources, and conductor doesn't know if a compute will fail, so it can't reschedule at that point, | |
| 19:49:55 | dansmith | but if we have a reason, like we go create the instance in a cell, and then learn we lost the race, we can reschedule before we map that instance | |
| 19:50:51 | dansmith | you know, | |
| 19:52:10 | dansmith | I was going to spout off another potential way to solve the need for the affinity upcall in a similar way, but I need to think about that for a sec | |
| 19:52:29 | dansmith | but regardless for the cell race I think we can manage it in superconductor | |
| 19:54:51 | sean-k-mooney | dansmith: is there any reason we dont create placement aggregates for cells out of interest? | |
| 19:55:24 | melwitt | I hope somebody can write down these ideas somewhere so we don't forget in stein | |
| 19:55:41 | sean-k-mooney | if we did that consider a cell is just a member-of right? | |
| 19:56:14 | sean-k-mooney | all the info to create the aggregate is already in the cell mapping table | |
| 19:56:24 | dansmith | sean-k-mooney: I don't want to leak cells outside of nova in any way | |
| 19:56:44 | dansmith | sean-k-mooney: and I definitely don't want to auto-create aggregates in nova that align to cells | |
| 19:57:18 | sean-k-mooney | dansmith: aggregates in placement auto-created by nova you mean | |
| 19:57:54 | dansmith | sean-k-mooney: definitely not auto-created in nova, but probably not really in placement either | |
| 19:59:35 | dansmith | we can already filter hosts by cell in the scheduler | |
| 19:59:36 | sean-k-mooney | well a placement aggreate is just a group of resouce providers. there is no real inherint semantics beyond that. so you can model cells or regions or host aggregats equally | |
| 20:00:23 | sean-k-mooney | dansmith: true but if there was a agregate associated with placement could ignore everything not in that aggregate | |
| 20:00:44 | sean-k-mooney | anyway its just an idea. | |
| 20:00:55 | openstackgerrit | Merged openstack/nova master: Refactor cell_type in compute/api.py https://review.openstack.org/589833 | |
| 20:21:07 | melwitt | mriedem: don't forget to please review these ocata quota fixes before you go on vacay https://review.openstack.org/588416 and the one above it | |
| 20:22:17 | mriedem | oh right | |
| 20:39:46 | openstackgerrit | melanie witt proposed openstack/nova master: Add a prelude release note for the 18.0.0 Rocky GA https://review.openstack.org/589303 | |
| 20:51:13 | openstackgerrit | Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548 | |
| 21:07:00 | openstackgerrit | Sergii Golovatiuk proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548 | |
| 21:13:30 | openstackgerrit | Jay Pipes proposed openstack/nova master: split gigantor SQL placement query into multiple https://review.openstack.org/590041 | |
| 21:16:14 | jaypipes | mriedem, dansmith: is there a way to boot a Nova VM on behalf of another user/project? | |