| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-06 | |||
| 14:27:59 | mriedem | i can hack something up for the nova side blocker | |
| 14:28:01 | artom | And I then I don't think we'd need a "3. make cinder handle 2" because we'd never get there | |
| 14:28:41 | smcginnis | mriedem + k8s == snark :) | |
| 14:33:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Block swap volume with multiattach volumes https://review.openstack.org/572790 | |
| 14:33:18 | mriedem | there ^ - afk for a couple of hours for a kid's school thing | |
| 14:34:57 | dansmith | mriedem: back for cells meeting? | |
| 14:35:40 | artom | mriedem, https://bugzilla.redhat.com/show_bug.cgi?id=1446446 <--- downstream bug with direct call to volume update | |
| 14:36:05 | openstack | artom: Error: Could not parse XML returned by bugzilla.redhat.com: Unknown host. (https://bugzilla.redhat.com/show_bug.cgi?id=1446446&ctype=xml) | |
| 14:36:20 | artom | openstack, ok :) | |
| 14:36:36 | artom | (It just took me forever to find it) | |
| 14:41:42 | mnaser | dansmith: regarding the build failure limit change, nova does include all weighers by default, right? | |
| 14:41:54 | dansmith | mnaser: yup | |
| 14:42:20 | dansmith | mnaser: you adjust the weight to make a weigher a no-op | |
| 14:42:46 | mnaser | dansmith: cool, with that, its lgtm, there are a few copy-paste thing but it's not a big deal, only if you have to do another revision you can fix it i guess | |
| 14:43:21 | dansmith | mnaser: ah, not copy-paste, but "mriedem decided he wanted to s/boot/build/" and I missed one :) | |
| 14:43:36 | dansmith | oh, | |
| 14:43:42 | dansmith | the ram one is copy pasta yeah | |
| 14:43:58 | mnaser | :P | |
| 14:44:07 | dansmith | I'll update so it's clean since this is backportable | |
| 14:44:08 | dansmith | plus nobody has really looked yet :/ | |
| 14:45:10 | mnaser | your call :) | |
| 14:45:18 | dansmith | mnaser: | |
| 14:45:19 | openstackgerrit | Dan Smith proposed openstack/nova master: Change consecutive build failure limit to a weigher https://review.openstack.org/572195 | |
| 14:46:21 | mnaser | dansmith: awesome, thank you very much for your work on that | |
| 14:46:27 | dansmith | np | |
| 14:54:14 | efried | bauzas, jaypipes: I think https://review.openstack.org/#/c/560459/ is ready when you get a chance. I'm not +2ing because I contributed. | |
| 14:54:24 | bauzas | ok | |
| 15:01:52 | sahid | stephenfin: thanks for the reviex on https://review.openstack.org/#/c/553072/ | |
| 15:04:38 | bauzas | dansmith: just saw https://review.openstack.org/#/c/572195/ | |
| 15:05:09 | bauzas | dansmith: fortunately to https://review.openstack.org/#/c/572195/4/nova/scheduler/host_manager.py, it will work with queens computes | |
| 15:05:35 | stephenfin | sahid: no problem | |
| 15:05:37 | dansmith | bauzas: right that's the point | |
| 15:06:11 | bauzas | dansmith: but https://review.openstack.org/#/c/572195/4/nova/conf/scheduler.py is asking for a default number that's huge | |
| 15:06:33 | dansmith | bauzas: did you read the justification for that number in the commit message? | |
| 15:06:45 | bauzas | sec, I maybe missed it | |
| 15:06:53 | bauzas | if so, my bad | |
| 15:10:06 | dansmith | if it's wrong, please do call it out, but it _was_ an intentional plan :) | |
| 15:10:29 | bauzas | dansmith: /me looking at the related bug to understand the problem | |
| 15:12:23 | dansmith | the first paragraph pretty much sums it up, but the related bug is also good context | |
| 15:14:45 | bauzas | dansmith: humpf, I wasn't knowing https://github.com/openstack/nova/commit/f93f675a :) | |
| 15:15:07 | dansmith | really? okay :) | |
| 15:15:15 | bauzas | yeah, really | |
| 15:15:25 | dansmith | you were in boston, no? | |
| 15:15:44 | bauzas | ok, so now I understand why we want to shuffle computes per number of failed builds | |
| 15:16:06 | bauzas | dansmith: yup, sorry but I don't remember that discussion :( | |
| 15:16:08 | openstackgerrit | Curt Moore proposed openstack/nova-specs master: Add spec for downloading images via RBD https://review.openstack.org/572805 | |
| 15:16:11 | dansmith | okay | |
| 15:16:30 | bauzas | at least, I now understand the problem | |
| 15:21:02 | bauzas | dansmith: in general, we said a couple of times to operators to randomize more the computes by using host_subset_size or shuffle_best_same_weighed_hosts | |
| 15:21:20 | bauzas | dansmith: so that a new instance was asking for a separate compute | |
| 15:21:30 | dansmith | yeah, but people that want strict packing might not want that | |
| 15:21:38 | bauzas | dansmith: agreed | |
| 15:21:45 | jaypipes | efried: done. | |
| 15:21:46 | bauzas | asking to pack is a problem | |
| 15:21:54 | efried | jaypipes: thanks | |
| 15:22:13 | bauzas | dansmith: what I was more thinking was maybe to deprecate one of the two options I mentioned | |
| 15:22:23 | bauzas | dansmith: and recommend using your own weigher | |
| 15:22:47 | bauzas | because once we merge your weigher, we'll get 3 opts for quite the same concern | |
| 15:23:06 | dansmith | um, why? | |
| 15:23:10 | dansmith | I don't think those overlap | |
| 15:23:40 | bauzas | dansmith: I think shuffle_best_same_weighed_hosts overlaps | |
| 15:24:07 | dansmith | with my weigher? | |
| 15:24:20 | bauzas | dansmith: it's two different implementations | |
| 15:24:27 | dansmith | I don't get that | |
| 15:24:38 | bauzas | but at the end, we want to make sure we pack correctly | |
| 15:25:10 | dansmith | without this weigher, hosts that have been failing will be considered at the same score as those that haven't.. shuffle just changes the likelihood you'll land on one of those, modulo the subset size | |
| 15:25:26 | dansmith | with my weigher they're likely not in consideration, moving your chance closer to 100% | |
| 15:26:00 | bauzas | right, that's my point | |
| 15:26:17 | bauzas | asking to shuffle is just because you want to make sure you go elsewhere | |
| 15:26:35 | dansmith | so, without this weigher, | |
| 15:26:44 | dansmith | lets say your subset size is 3 | |
| 15:26:48 | dansmith | and shuffle is turned on | |
| 15:27:09 | dansmith | if you have a few hosts that are empty, they score high. if one of those is misconfigured, | |
| 15:27:18 | dansmith | your chance of booting is 66% | |
| 15:27:41 | dansmith | with this weigher, you'd get two empty hosts, plus one that has stuff on it in the subset | |
| 15:27:44 | bauzas | right, I'm saying your weigher improves the changes | |
| 15:27:47 | bauzas | chances* | |
| 15:28:06 | bauzas | that's why I don't see the need for shuffle_best_same_weighed_hosts | |
| 15:28:25 | bauzas | I'm asking to deprecate shuffle_best_same_weighed_hosts | |
| 15:28:49 | bauzas | because this option just does a random shuffle | |
| 15:28:52 | dansmith | but, in the absence of failing hosts, shuffle still helps to balance concurrent boot traffic right? | |
| 15:29:04 | dansmith | I don't see shuffle as being used for avoiding faults at all | |
| 15:32:07 | bauzas | maybe my concern is just that I dislike this shuffle option :) | |
| 15:32:46 | bauzas | if you want to spread on a specific metric, just ask for a weigher using that metric | |
| 15:32:57 | bauzas | instead of just hoping you'll get spread for free | |
| 15:32:59 | bauzas | anyway | |
| 15:34:45 | dansmith | shuffle is purely to avoid super strict packing right? | |
| 15:34:49 | dansmith | if you have big computes and small instances, | |
| 15:35:04 | dansmith | you might build 100 instances in the same exact compute before it's completely full and you move onto the next bucket in the line | |
| 15:35:21 | dansmith | shuffle seems good for just saying "we generally want packing, but not suuuuuper strict" | |
| 15:38:37 | bauzas | dansmith: "life, uh, always finds a way". Replace "life" by "weigher" and you'll get my mind here :) | |
| 15:39:00 | bauzas | anyway, back to review | |
| 15:43:10 | openstackgerrit | Mathieu Gagné proposed openstack/nova-specs master: Update multiple fixed-IPs support with services field https://review.openstack.org/572814 | |
| 15:46:52 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081 | |
| 15:55:28 | mriedem | dansmith: for the follow up https://review.openstack.org/#/c/572814/ | |
| 15:56:40 | dansmith | mriedem: don't we have to wait two years first? | |
| 15:56:47 | dansmith | like fine aged cheddar | |
| 15:57:04 | mriedem | artom linked a bz i don't have access to | |
| 16:00:00 | dansmith | mriedem: I looked and I don't think it has any useful context in it | |
| 16:00:54 | mriedem | ok | |