| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-21 | |||
| 14:25:06 | smcginnis | mriedem: Decising which sessions to accept for the official schedule. | |
| 14:25:09 | johnthetubaguy | mriedem: done it twice, its a pain | |
| 14:25:15 | mriedem | oh, i don't care about that session then :) | |
| 14:25:17 | smcginnis | johnthetubaguy: Yep. :) | |
| 14:25:21 | alex_xu | mriedem: hope you didn't start that | |
| 14:25:29 | mriedem | alex_xu: no i didn't, thanks for doing it | |
| 14:25:34 | alex_xu | mriedem: np | |
| 14:29:16 | mriedem | alex_xu: comments inline :) | |
| 14:29:19 | mriedem | otherwise looks good | |
| 14:30:36 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Always disconnect_volume after rebase failures https://review.openstack.org/458807 | |
| 14:31:37 | mriedem | bauwser: these https://review.openstack.org/#/c/457353/ https://review.openstack.org/#/c/448235/ https://review.openstack.org/#/c/458668/ https://review.openstack.org/#/c/456770/ https://review.openstack.org/#/c/448253/ | |
| 14:31:52 | bauwser | mriedem: reviewy reviewy | |
| 14:32:03 | mriedem | ask and ye shall receieve | |
| 14:32:05 | mriedem | *receive | |
| 14:32:26 | bauwser | mriedem: FWIW, I haven't touched the scheduler claims spec since I need time to think about | |
| 14:32:51 | mriedem | i haven't either | |
| 14:32:52 | bauwser | about the super extra cool problem you showed to the world with overhead values | |
| 14:33:04 | mriedem | don't thank me, thank some performance tester internal to huawei | |
| 14:33:33 | bauwser | mriedem: FWIW, my honest first opinion is that problem it's something that we should leave out of placement | |
| 14:33:35 | mriedem | i didn't know the overhead stuff existed before that | |
| 14:33:58 | bauwser | mriedem: that looks something not really useful for placement and very related to the hypervisor | |
| 14:34:28 | mriedem | you have to solve it with placement one way or the other, either by tracking overhead in allocations, or forcing extra reserved inventory to make up for the overhead | |
| 14:34:29 | bauwser | so, say we leave in a world where we're not trying to eat all the resources but still have reserved room for that, then it's not a problem for placement | |
| 14:34:30 | mriedem | right? | |
| 14:34:52 | mriedem | i don't think we can punt on this | |
| 14:35:55 | bauwser | mriedem: if we have rescheduling, would it be a problem ? | |
| 14:36:14 | bauwser | we already don't place instances by reviewing that value | |
| 14:36:25 | mriedem | we reschedule based on it though | |
| 14:36:29 | bauwser | of course | |
| 14:36:31 | mriedem | but we don't have reschedules with super conductor | |
| 14:36:34 | bauwser | but how much ? | |
| 14:36:38 | mriedem | idk how much | |
| 14:36:47 | bauwser | so, it's rather a problem about how to reschedule | |
| 14:36:53 | mriedem | well, | |
| 14:36:59 | mriedem | the point of claims in the scheduler is you don't need to reschedule | |
| 14:37:17 | bauwser | honestly, I think claiming in the scheduler couldn't remove the compute node verifying the resources too | |
| 14:37:31 | bauwser | because sometimes, things can be wrong | |
| 14:37:57 | bauwser | so, having claims in the scheduler is cool but like Pareto (or more) fixing 95% of the number of problems we have | |
| 14:38:08 | bauwser | should we fix 5% of the current problems by adding more ? | |
| 14:38:15 | bauwser | if we still reschedule | |
| 14:38:18 | mriedem | if you can't reschedule from the compute, then what? NoValidHost and put the instance into ERROR state? | |
| 14:38:35 | mriedem | ^ was one of the options i proposed in the spec | |
| 14:38:44 | bauwser | anyway, since not all the resources are yet set in the placement API (like NUMA), we would still need to reschedule, right ? | |
| 14:38:54 | mriedem | i said account for the overhead by updating the allocations once on the compute, but people didn't like that b/c of quotas | |
| 14:39:19 | bauwser | because we use allocations for quotas, and I agree with the majority | |
| 14:39:42 | bauwser | so, either we say we add that for inventories *and not allocations*, or we say it's not a problem | |
| 14:39:54 | bauwser | and we leave reschedules fix that self-healing | |
| 14:40:07 | mriedem | we don't have reschedules | |
| 14:40:29 | bauwser | that's what I'm not sure we can just say it won't exist in a cellsv2 world | |
| 14:40:29 | mriedem | i thought the consensus the other day was pad the RP inventory reserved amount | |
| 14:40:37 | mriedem | via the get_inventory() method on drivers that calculate overhead | |
| 14:40:41 | bauwser | mriedem: I think the consensus looks good by me | |
| 14:40:53 | mriedem | reschedules are possible in a single-cell deployment | |
| 14:40:54 | bauwser | that's my "either" possibiltyu | |
| 14:41:05 | mriedem | but i don't think we should rely on them | |
| 14:41:09 | bauwser | okay | |
| 14:41:11 | mriedem | when designing things | |
| 14:41:34 | mriedem | dan proposed adding a config option to disable reschedules so you never have upcalls, regardless of deployment | |
| 14:41:44 | bauwser | so it leaves the idea to modify inventories to add more "something" for decrementing the "left" value | |
| 14:42:25 | bauwser | amending 'reserved' could be that "something" which wouldn't need to add a new DB fiedl | |
| 14:42:50 | mriedem | correct, that's what most people wanted the other day | |
| 14:42:58 | bauwser | mriedem: yup, and I +2d that change for the instance scheduler RPC call | |
| 14:43:06 | mriedem | it sucks for the operator because they have to configure reserved amounts, | |
| 14:43:13 | mriedem | unless we can come up with a scheme to automate it | |
| 14:43:14 | bauwser | mriedem: okay, if so, I think I can amend that spec | |
| 14:43:44 | bauwser | mriedem: I was rather thinking to have the inventory be modified by the RT to augment it by the sum of overhead for each instance | |
| 14:43:46 | openstackgerrit | Mikhail Feoktistov proposed openstack/nova master: Remove fstype param from ploop init https://review.openstack.org/444970 | |
| 14:43:53 | bauwser | snap | |
| 14:44:01 | bauwser | to have the inventory.reserved value | |
| 14:44:02 | cfriesen | question...has nova ever considered setting the virtual disk logical/physical block size based on the host disk block size? currently the guest sees 512 even for disks with 4K block size, which potentially leads to extra read-modify-write overhead | |
| 14:44:40 | leakypipes | mriedem: k, on it. | |
| 14:46:23 | bauwser | leakypipes: mriedem: okay, I think then I have enough to amend the spec | |
| 14:46:44 | leakypipes | bauwser: coolio. | |
| 14:46:44 | bauwser | I'll try to provide a new rev' by today EOB or Monday | |
| 14:46:55 | mriedem | bauwser: so rather than add overheads to allocations, you want to add overhead to inventory.reserved during each build? | |
| 14:47:28 | mriedem | that could still get you in a 409 situation where we have to fail the build | |
| 14:47:31 | bauwser | leakypipes: do you agree with my idea to augment in the RT the reserved value given by the driver.get_inventory() method by the number of instance.overhead we have ? | |
| 14:48:19 | mriedem | we had talked about just a configurable value that get_inventory() would provide and isn't per-instance | |
| 14:48:47 | figleaf | I don't like treating reserved like an allocation - it should be set when the RP is created, and that's it | |
| 14:48:49 | leakypipes | bauwser: hmm, I must have missed that. lemme have another look at the spec | |
| 14:48:58 | bauwser | leakypipes: that's not yet in the spec | |
| 14:48:59 | mriedem | leakypipes: it's new as of bauwser today | |
| 14:49:02 | leakypipes | oh | |
| 14:49:05 | mriedem | last 5 minutes | |
| 14:49:13 | figleaf | it's sort of in my last comment on the spec | |
| 14:49:17 | leakypipes | no wonder I missed it then :) | |
| 14:49:22 | bauwser | I'm trying to explain that we find nodes based on what we ask vs. what we have as free | |
| 14:49:32 | bauwser | so, what we ask is the flavor | |
| 14:49:44 | bauwser | what we have is the inventory minus the allocations | |
| 14:49:48 | leakypipes | bauwser: sorry, lemme read back up. I just got back from breakfast :) | |
| 14:50:04 | bauwser | I'm trying to explain it another way | |
| 14:50:27 | bauwser | given we ask for the flavor size | |
| 14:50:34 | mriedem | reserved to me means static reserved, via config from the operator, something that nova doesn't mess with automatically | |
| 14:50:49 | bauwser | and we compare it to each node by doing (total - reserved - sum(allocations)) | |
| 14:51:00 | figleaf | mriedem: exactly | |
| 14:51:15 | bauwser | I'm seeing some way to say (total - reserved - sum(allocations) - sum(overhead)) | |
| 14:51:26 | figleaf | for a hypervisor, it's the "cost of doing business" | |
| 14:51:32 | mriedem | worst case scenario is you don't modify reserved at all and run the risk of over-consuming your resources on the host | |
| 14:51:47 | bauwser | my proposal being "reserved" and sum(overhead) is purely related to the hypervisor | |
| 14:51:48 | mriedem | for ram and disk it could be a small overhead | |