| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-23 | |||
| 12:59:46 | stephenfin | without my subunit change, it craps out with the subunit parser error | |
| 12:59:55 | stephenfin | with it, it still craps out but with a proper error | |
| 13:00:04 | jawad_axd | Quick question guys: What does cpu allocation 2.0 means ? Is it 2:1? | |
| 13:00:16 | stephenfin | saying the packet is > 4k (I think) | |
| 13:01:06 | jawad_axd | and it is safe to change cpu allocation ratio in running environment? | |
| 13:01:14 | stephenfin | sorry, 4M https://github.com/testing-cabal/subunit/blob/master/python/subunit/v2.py#L202-L208 | |
| 13:01:43 | stephenfin | jawad_axd: yup, 2:1 (20 enabled host CPUs = 40 VCPU inventory) | |
| 13:02:18 | stephenfin | jawad_axd: It should be safe to change so long as you don't lower it to the point that there's less inventory available than you have used | |
| 13:02:56 | stephenfin | I'm not actually sure what would happen then. I assume the periodic task to update placement's inventory would start failing | |
| 13:03:18 | stephenfin | easily tested in a pre-prod environment :) | |
| 13:03:41 | sean-k-mooney | stephenfin: ok well if we have a repoducer that means we have a chance of fixing it | |
| 13:03:57 | sean-k-mooney | we dont need nova at all we can repodcuse this in stestr | |
| 13:04:02 | stephenfin | Exactly. That's pretty damn consistent | |
| 13:04:20 | stephenfin | We should be able to but I couldn't do so when I tried | |
| 13:04:29 | stephenfin | Probably didn't have the correct fixtures configured or something | |
| 13:05:10 | sean-k-mooney | hum ok well i might take a look at this later. and see if i can figure something out | |
| 13:05:33 | sean-k-mooney | if we can create a simpler repoducer that woudl be good if not this works | |
| 13:06:58 | jawad_axd | @stephenfin Thanks. One more thing, In horizon I see cpu's under compute(hypervisor ) tab as 24cpu. So with 2.0 cpu allocation ratio, I should be able to use 24x2=48 vcpus, right? But I am only able to use 24 cpus.What you say about it? | |
| 13:07:42 | stephenfin | I imagine Horizon is pulling that info from the os-hypevisor API which doesn't take overcommit ratios into account | |
| 13:07:53 | sean-k-mooney | i think so too | |
| 13:08:03 | sean-k-mooney | i think horizon is showing the correct value | |
| 13:08:29 | sean-k-mooney | the allocation raitio can be change per host it would not be resonable to have to compute it differnetly per hosts | |
| 13:09:53 | jawad_axd | If horizon is showing the same cpu's on host. Does it mean, overcommitment is not being used/applied ? | |
| 13:10:04 | sean-k-mooney | no | |
| 13:10:23 | sean-k-mooney | over commit is calulated in the scheduler/placment | |
| 13:10:32 | sean-k-mooney | it should not be see in horizon | |
| 13:10:36 | stephenfin | what sean-k-mooney said | |
| 13:10:55 | stephenfin | Horizon might go into negative available values (I'm not sure) but instances will still be scheduled | |
| 13:11:32 | sean-k-mooney | jawad_axd: if you are using a recent version of openstack and look at the placment RP for the host you will see an cpu inventory where the total = the number of cores on the host and the allocation ratio will be 2.0 | |
| 13:11:55 | sean-k-mooney | well on my home system it currently shos 44/24 | |
| 13:12:10 | sean-k-mooney | so it does not go into negitiv but the used can exceed the available | |
| 13:12:19 | jawad_axd | I am just stuck because I can not use more than 24 cpus, while with overcommitment I should be able to use 48. | |
| 13:12:38 | jawad_axd | I am using stein | |
| 13:12:42 | sean-k-mooney | you cannot use more the 24cpu in one guest or in general | |
| 13:13:05 | jawad_axd | In general. | |
| 13:13:09 | sean-k-mooney | did you set the cpu allcoation ratio in the compute node config | |
| 13:13:11 | stephenfin | also, how are you creating the instance? Via the Horizon UI or on the CLI? | |
| 13:13:25 | jawad_axd | cpu_allocation_ratio = 2.0 | |
| 13:13:25 | jawad_axd | # Scheduler | |
| 13:13:36 | sean-k-mooney | ya that does not work in stien | |
| 13:13:43 | jawad_axd | From both, cli and horizon | |
| 13:13:44 | sean-k-mooney | you hage to set it per compute node | |
| 13:13:55 | jawad_axd | ah ok. | |
| 13:14:13 | jawad_axd | how to set per compute node? Can you give me some pointers plz? | |
| 13:14:46 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.cpu_allocation_ratio | |
| 13:15:00 | sean-k-mooney | you just set cpu_allocation_ratio in the default section | |
| 13:15:09 | sean-k-mooney | in the nova.conf | |
| 13:15:35 | sean-k-mooney | alternitivly if you want to manage it via the placement api you can use initial_cpu_allocation_ratio | |
| 13:18:38 | jawad_axd | So cpu_allocation_ratio in nova.conf at each compute node will work for that compute node? | |
| 13:18:58 | sean-k-mooney | correct it should | |
| 13:19:24 | sean-k-mooney | what i belive is happening is that the allcoation raition in the placement inventory is set to 1 | |
| 13:19:43 | sean-k-mooney | and that is causeign the host to be eliminated before you get to the schduler | |
| 13:19:47 | sean-k-mooney | where you set it to 2 | |
| 13:19:58 | sean-k-mooney | you could check that | |
| 13:20:19 | sean-k-mooney | you can do "openstack resource provider list" | |
| 13:20:48 | sean-k-mooney | then do "openstack resource provider inventory show <RP uuid>" | |
| 13:20:57 | rouk | cant use 24 vcore or more per guest? when was this? i have 32vcore guests. | |
| 13:21:51 | sean-k-mooney | you can never have more cores in the guest then are on the host but you can have multple guests sharing cores on the host | |
| 13:22:42 | sean-k-mooney | so on my home server i currently have 44 guest cpus spread over 20 host cpus | |
| 13:22:49 | sean-k-mooney | but i could not boot a 44 core vm | |
| 13:23:51 | jawad_axd | @sean-k-mooney ok | |
| 13:29:35 | jawad_axd | @sean-k-mooney Yeah "openstack resource provider inventory show 04b5aba0-ca20-4e15-881b-eaca3ce73db7 VCPU" show overcommitment 24x2=48. So probably, I was using more than 24cpus for one guest which I cant do as you said. | |
| 13:33:03 | sean-k-mooney | jawad_axd: there are some other constrats as well | |
| 13:33:12 | sean-k-mooney | if you are using cpu pinning you cannot over commit | |
| 13:33:41 | sean-k-mooney | and if you are using a virtual numa toplgoy then within each numa node the cpus cant be over commited | |
| 13:35:01 | jawad_axd | ok my hosts have mostly this architecture, Can you have a look at it plz. http://paste.openstack.org/show/788715/ | |
| 13:36:14 | sean-k-mooney | ya so you have 4 numa nodes and 12 cores per numa node | |
| 13:36:40 | sean-k-mooney | well 12 threads | |
| 13:36:53 | jawad_axd | Does it allow overcommitment ? | |
| 13:37:04 | sean-k-mooney | sure | |
| 13:37:10 | sean-k-mooney | but it depends on the vm | |
| 13:37:28 | sean-k-mooney | if the vm has a numa toplogy it impacts how the vm can be placed | |
| 13:37:30 | jawad_axd | If vms are using cpu host-model, then? | |
| 13:37:37 | sean-k-mooney | no | |
| 13:37:50 | rouk | its a flag, since numa is not at all free. | |
| 13:37:56 | sean-k-mooney | if the vm flavor has hw:cpu_polciy=dedeicated | |
| 13:38:19 | sean-k-mooney | hw:mem_page_size or hw:numa_nodes set or is using vPMEM | |
| 13:38:27 | jawad_axd | I am not using numa virtually. | |
| 13:38:30 | sean-k-mooney | then it will have a numa toplogy | |
| 13:38:58 | sean-k-mooney | in that case only the gloabl host core count and allcoation ration for that host matter | |
| 13:39:12 | jawad_axd | So I guess, it should be fine for me then. | |
| 13:39:20 | sean-k-mooney | yep | |
| 13:39:30 | jawad_axd | Thanks !!! | |
| 13:40:16 | sean-k-mooney | if you set hw:numa_nodes=1 you are basicaly saying i want a guest with 1 numa node and map it to a single host numa node. if you enbale hugepage or cpu pinning it imples hw:numa_nodes=1 | |
| 13:40:44 | sean-k-mooney | so with that toploy you would only be able to have at most 12 cores in a vm | |
| 13:41:00 | sean-k-mooney | but since you are not using numa for the vms it does not matter in your case | |
| 13:41:18 | jawad_axd | Okay.. | |
| 13:42:01 | sean-k-mooney | the numa feature are just for performance tuning | |
| 13:42:10 | sean-k-mooney | if you dont need them you can ignore them | |
| 13:42:26 | jawad_axd | I dont need them now, maybe in future. | |
| 13:59:17 | efried | nova meeting now-ish | |
| 13:59:49 | shilpasd | efried: hi | |
| 13:59:53 | efried | hello | |
| 14:03:50 | shilpasd | efried: dansmith: here added one question, https://review.opendev.org/#/c/694462/8/nova/db/sqlalchemy/api.py@2008 | |
| 14:04:16 | shilpasd | help me to understand 'any issues in setting 'volume_backed' field on the existing instances in the upgrade db script?' | |
| 14:05:13 | efried | shilpasd: dansmith is on vacation this week, and I'm not really qualified to discuss this one. It's Chinese New Year around now, so I'm not sure if alex_xu is around either. | |
| 14:05:23 | efried | IOW this may have to wait until next week to close on. | |
| 14:05:50 | shilpasd | efried: ok, thanks | |
| 14:06:13 | shilpasd | will wait for them | |
| 14:12:32 | sean-k-mooney | well that was quick :) | |