| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-19 | |||
| 14:58:32 | edleafe | mriedem: I mean when creating the resource provider, add the total overhead it might need to the reserved part for the hypervisor | |
| 14:58:33 | sfinucan | bauzas: Correct | |
| 14:58:37 | cdent | mriedem: that's not what I said. What I said was _if_ we change reserved, we don't change the allocations. I actually think making a good guess at what global overhead is at boot time, and adding that to reserved is the right way to go | |
| 14:58:56 | cdent | s/boot/host boot/ | |
| 14:59:01 | sfinucan | We call 'numa_fit_instance_to_host' to make sure that the instance would fit on a host with a given CPU pinning permutation | |
| 14:59:09 | dansmith | cdent: meaning claim more and update later? | |
| 14:59:31 | mriedem | edleafe: cdent: i don't really know how we can safely estimate that, | |
| 14:59:31 | edleafe | dansmith: no, just claim what the instance uses | |
| 14:59:42 | dansmith | edleafe: eh? | |
| 14:59:44 | mriedem | w/o knowing which flavors will be used, which scheduler filters are being used, etc | |
| 14:59:46 | cdent | dansmith: no, continue making allocations using the non-overhead values. incorporate the expected in 'reserved' of inventory | |
| 14:59:47 | sfinucan | It's the easiest way to check that out for things like cpu_thread_policy, where we can't do a simple "have I X free CPUs" check | |
| 14:59:51 | dansmith | edleafe: we're claiming before we know that, that's the whole point | |
| 15:00:02 | bauzas | okay, here is a thinh | |
| 15:00:04 | edleafe | dansmith: it would already be accounted for | |
| 15:00:09 | dansmith | cdent: oh, that's what I initially was saying, yeah | |
| 15:00:12 | bauzas | given all the nasty and fancy things we do with RT | |
| 15:00:15 | edleafe | in the RP's reserved | |
| 15:00:17 | bauzas | I have a though | |
| 15:00:38 | bauzas | why not just posting an allocation without all that stuff, and do modify the allocation at the RT state ? | |
| 15:00:43 | dansmith | cdent: not claim the flavor+overhead, but claim flavor, and require reserved to be set high enough to cover it | |
| 15:00:53 | mriedem | bauzas: that's one of the options i said in the spec | |
| 15:00:56 | dansmith | bauzas: because that's terrible | |
| 15:01:00 | edleafe | I see chasing the last tiny bit of resource is going to make the code an even bigger mess than it is now | |
| 15:01:01 | bauzas | ie. we claim for the resouces we know, but we later heal that when we claim | |
| 15:01:17 | bauzas | when we /RT/.claim | |
| 15:01:21 | cdent | dansmith: yes, but doing that in an semi-automated way based on the initial calculated (and config'd) inventory. Updating it at boot time. | |
| 15:01:32 | bauzas | either way, placement doesn't know a bit of that NFV gangband | |
| 15:01:38 | dansmith | cdent: well, whether or not we do that is a detail I guess | |
| 15:01:53 | bauzas | so I think NFV claims would still be a 2-phase step | |
| 15:02:07 | dansmith | cdent: however, I think I lost sight of that as a simple option early on | |
| 15:02:14 | bauzas | ie. rounding the allocation by the scheduler, and refine that allocation as soon as we have a better vision | |
| 15:02:18 | dansmith | cdent: that's probably the way to go, yeah, and do the more complex thing if that ends up being to naive for some reason | |
| 15:02:24 | cdent | dansmith: yes | |
| 15:02:39 | cdent | otherwise we run into some very complex wangling for unclear gains | |
| 15:02:41 | dansmith | cdent: I lost sight of it when we started talking about claiming the flavor+overhead in one go | |
| 15:02:48 | dansmith | cdent: which means we couldn't rely on reserved | |
| 15:03:07 | bauzas | soooo | |
| 15:03:36 | mriedem | dansmith: cdent: so would that be based on reserved_host_disk_mb and reserved_host_memory_mb ? | |
| 15:03:38 | bauzas | could we save my life, and assume that we would POST an allocation that is rounded, and later refine it within the instance_claim phase, exactly like we already do ? | |
| 15:04:09 | bauzas | 90% of users would have benefits, and things wouldn't change (even improve) for NFV users | |
| 15:04:10 | cdent | bauzas: we slightly increase the risk of retry/error if the second claim can't claim | |
| 15:04:22 | dansmith | mriedem: cdent: and vcpu | |
| 15:04:24 | cdent | seems better to not allocation overhead per instance | |
| 15:04:37 | bauzas | cdent: like we already do :) | |
| 15:04:41 | cdent | yes | |
| 15:04:46 | cdent | and instead cover it with reserved | |
| 15:04:48 | cdent | it is safer | |
| 15:04:52 | dansmith | mriedem: cdent: meaning, we'd have to always have at least one reserved vcpu | |
| 15:04:59 | cdent | more likely to have successful claims more often | |
| 15:05:08 | mriedem | dansmith: today in _compute_node_to_inventory_dict in the scheduler report client we don't reserve any vcpu | |
| 15:05:15 | dansmith | right | |
| 15:05:19 | cdent | dansmith: I believe you, but why? | |
| 15:05:33 | dansmith | cdent: because libvirt needs to have vcpu+1 if certain flavor configs are set | |
| 15:05:39 | cdent | ah, okay | |
| 15:06:08 | dansmith | cdent: if you ask for dedicated io threads or something, then it requires another vcpu dedicated to you to handle that | |
| 15:06:12 | mriedem | dansmith: cdent: thinking ahead, _compute_node_to_inventory_dict is only used for the old style driver.get_available_resource call, | |
| 15:06:24 | dansmith | cdent: which is really expensive, and probably unfortunate for people that don't want that, so we'd need a way to not reserve the last vcpu on a system for people that don't want it | |
| 15:06:28 | mriedem | with get_inventory() we could make the reservation calculation in the virt drivers that actually do this overhead thing | |
| 15:06:55 | cdent | yes | |
| 15:07:06 | dansmith | sure, but libvirt needs to not do that if the operator doesn't want it, | |
| 15:07:17 | dansmith | because they may have no flavors configured for this dedicated thing, | |
| 15:07:23 | dansmith | and we don't want to prevent them from using their last vcpu :) | |
| 15:07:45 | mriedem | are you talking to me or cdent? | |
| 15:07:49 | cdent | how much does that matter (the last vcpu) | |
| 15:07:56 | cdent | mriedem: yes, about get_inventory, I like get_inventory | |
| 15:07:56 | dansmith | you | |
| 15:08:08 | dansmith | heh | |
| 15:08:09 | dansmith | cdent: vcpus are few | |
| 15:08:10 | dansmith | cdent: compared to memory or disk | |
| 15:08:15 | bauzas | sorry, in a meeting atm | |
| 15:08:24 | dansmith | cdent: imagine a cloud of 4-core ARM boxes | |
| 15:08:32 | dansmith | where you lose one core per system | |
| 15:08:54 | cdent | are we losing a pcpu or a vcpu? | |
| 15:09:04 | mriedem | dansmith: "libvirt needs to not do that if the operator doesn't want it," - we just take that into account in the libvirt get_inventory() method don't we? | |
| 15:09:15 | mriedem | when determining the reserved value for the VCPU resource provider inventory? | |
| 15:09:16 | cdent | (physical, not pinned) | |
| 15:09:16 | dansmith | mriedem: yeah, it just has to be configurable | |
| 15:09:25 | dansmith | mriedem: and we don't have that currently, AFAIK | |
| 15:09:32 | dansmith | mriedem: we have reserved disk and memory only right? | |
| 15:09:39 | mriedem | dansmith: as far as i know | |
| 15:09:46 | dansmith | cdent: pcpu in the case of \this dedicated thing | |
| 15:09:46 | mriedem | but i learn new terrible things every day :) | |
| 15:10:38 | cdent | dansmith: gotchya | |
| 15:11:00 | namnh | Hi everyone, I am reading the rolling upgrade in Nova, there is a point that make me confuse. According to the docs, we have to gracefully shutdown all Nova services except nova-compute. In my understanding, this will ensures no old server (like nova-api, nova-conductor, nova-scheduler) won't interact with new schema DB. | |
| 15:11:12 | namnh | But in the docs was written "Install the code for the next version of Nova, either in a venv or a separate control plane node, including all the python dependencies. Using the newly installed nova code, run the DB sync. (nova-manage db sync; nova-manage api_db sync)." it means there will have period, old services interact with new schema DB. Could you please explain for me this point? | |
| 15:11:29 | namnh | Here is the docs: https://docs.openstack.org/developer/nova/upgrade.html#minimal-downtime-upgrade-process | |
| 15:11:36 | edleafe | dansmith: so those who are configuring flavors this way are already sacrificed a pcpu now, right? IOW, this wouldn't change it? | |
| 15:11:59 | edleafe | s/sacrificed/sacrificiing | |
| 15:12:14 | dansmith | namnh: nova compute does not talk to the db directly, so it won't touch the different schema | |
| 15:12:34 | dansmith | edleafe: they are, but only at claim time, not at reserved-for-the-future time | |
| 15:12:55 | dansmith | edleafe: since that is not config, but a flavor setting, we need a config to know if we _should_ reserve that or not | |
| 15:13:08 | edleafe | dansmith: ok, but in your 4 core example, they can only use 3 in either case | |
| 15:13:13 | dansmith | and of course, we should validate all this with the people that wrote this code :) | |
| 15:13:25 | dansmith | edleafe: but people that don't use that dedicated thing.. those are the ones we careabout | |
| 15:13:41 | dansmith | edleafe: because we don't want to reserve (and thus never schedule to) the fourth cpu | |
| 15:14:10 | edleafe | dansmith: so maybe add a config option to compute: WASTE_PCPU = True | |
| 15:14:17 | dansmith | edleafe: exactly what I'm saying | |
| 15:14:17 | edleafe | :) | |
| 15:14:58 | edleafe | and these are a small minority of deployments? | |