| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-23 | |||
| 13:10:55 | stephenfin | Horizon might go into negative available values (I'm not sure) but instances will still be scheduled | |
| 13:11:32 | sean-k-mooney | jawad_axd: if you are using a recent version of openstack and look at the placment RP for the host you will see an cpu inventory where the total = the number of cores on the host and the allocation ratio will be 2.0 | |
| 13:11:55 | sean-k-mooney | well on my home system it currently shos 44/24 | |
| 13:12:10 | sean-k-mooney | so it does not go into negitiv but the used can exceed the available | |
| 13:12:19 | jawad_axd | I am just stuck because I can not use more than 24 cpus, while with overcommitment I should be able to use 48. | |
| 13:12:38 | jawad_axd | I am using stein | |
| 13:12:42 | sean-k-mooney | you cannot use more the 24cpu in one guest or in general | |
| 13:13:05 | jawad_axd | In general. | |
| 13:13:09 | sean-k-mooney | did you set the cpu allcoation ratio in the compute node config | |
| 13:13:11 | stephenfin | also, how are you creating the instance? Via the Horizon UI or on the CLI? | |
| 13:13:25 | jawad_axd | # Scheduler | |
| 13:13:25 | jawad_axd | cpu_allocation_ratio = 2.0 | |
| 13:13:36 | sean-k-mooney | ya that does not work in stien | |
| 13:13:43 | jawad_axd | From both, cli and horizon | |
| 13:13:44 | sean-k-mooney | you hage to set it per compute node | |
| 13:13:55 | jawad_axd | ah ok. | |
| 13:14:13 | jawad_axd | how to set per compute node? Can you give me some pointers plz? | |
| 13:14:46 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.cpu_allocation_ratio | |
| 13:15:00 | sean-k-mooney | you just set cpu_allocation_ratio in the default section | |
| 13:15:09 | sean-k-mooney | in the nova.conf | |
| 13:15:35 | sean-k-mooney | alternitivly if you want to manage it via the placement api you can use initial_cpu_allocation_ratio | |
| 13:18:38 | jawad_axd | So cpu_allocation_ratio in nova.conf at each compute node will work for that compute node? | |
| 13:18:58 | sean-k-mooney | correct it should | |
| 13:19:24 | sean-k-mooney | what i belive is happening is that the allcoation raition in the placement inventory is set to 1 | |
| 13:19:43 | sean-k-mooney | and that is causeign the host to be eliminated before you get to the schduler | |
| 13:19:47 | sean-k-mooney | where you set it to 2 | |
| 13:19:58 | sean-k-mooney | you could check that | |
| 13:20:19 | sean-k-mooney | you can do "openstack resource provider list" | |
| 13:20:48 | sean-k-mooney | then do "openstack resource provider inventory show <RP uuid>" | |
| 13:20:57 | rouk | cant use 24 vcore or more per guest? when was this? i have 32vcore guests. | |
| 13:21:51 | sean-k-mooney | you can never have more cores in the guest then are on the host but you can have multple guests sharing cores on the host | |
| 13:22:42 | sean-k-mooney | so on my home server i currently have 44 guest cpus spread over 20 host cpus | |
| 13:22:49 | sean-k-mooney | but i could not boot a 44 core vm | |
| 13:23:51 | jawad_axd | @sean-k-mooney ok | |
| 13:29:35 | jawad_axd | @sean-k-mooney Yeah "openstack resource provider inventory show 04b5aba0-ca20-4e15-881b-eaca3ce73db7 VCPU" show overcommitment 24x2=48. So probably, I was using more than 24cpus for one guest which I cant do as you said. | |
| 13:33:03 | sean-k-mooney | jawad_axd: there are some other constrats as well | |
| 13:33:12 | sean-k-mooney | if you are using cpu pinning you cannot over commit | |
| 13:33:41 | sean-k-mooney | and if you are using a virtual numa toplgoy then within each numa node the cpus cant be over commited | |
| 13:35:01 | jawad_axd | ok my hosts have mostly this architecture, Can you have a look at it plz. http://paste.openstack.org/show/788715/ | |
| 13:36:14 | sean-k-mooney | ya so you have 4 numa nodes and 12 cores per numa node | |
| 13:36:40 | sean-k-mooney | well 12 threads | |
| 13:36:53 | jawad_axd | Does it allow overcommitment ? | |
| 13:37:04 | sean-k-mooney | sure | |
| 13:37:10 | sean-k-mooney | but it depends on the vm | |
| 13:37:28 | sean-k-mooney | if the vm has a numa toplogy it impacts how the vm can be placed | |
| 13:37:30 | jawad_axd | If vms are using cpu host-model, then? | |
| 13:37:37 | sean-k-mooney | no | |
| 13:37:50 | rouk | its a flag, since numa is not at all free. | |
| 13:37:56 | sean-k-mooney | if the vm flavor has hw:cpu_polciy=dedeicated | |
| 13:38:19 | sean-k-mooney | hw:mem_page_size or hw:numa_nodes set or is using vPMEM | |
| 13:38:27 | jawad_axd | I am not using numa virtually. | |
| 13:38:30 | sean-k-mooney | then it will have a numa toplogy | |
| 13:38:58 | sean-k-mooney | in that case only the gloabl host core count and allcoation ration for that host matter | |
| 13:39:12 | jawad_axd | So I guess, it should be fine for me then. | |
| 13:39:20 | sean-k-mooney | yep | |
| 13:39:30 | jawad_axd | Thanks !!! | |
| 13:40:16 | sean-k-mooney | if you set hw:numa_nodes=1 you are basicaly saying i want a guest with 1 numa node and map it to a single host numa node. if you enbale hugepage or cpu pinning it imples hw:numa_nodes=1 | |
| 13:40:44 | sean-k-mooney | so with that toploy you would only be able to have at most 12 cores in a vm | |
| 13:41:00 | sean-k-mooney | but since you are not using numa for the vms it does not matter in your case | |
| 13:41:18 | jawad_axd | Okay.. | |
| 13:42:01 | sean-k-mooney | the numa feature are just for performance tuning | |
| 13:42:10 | sean-k-mooney | if you dont need them you can ignore them | |
| 13:42:26 | jawad_axd | I dont need them now, maybe in future. | |
| 13:59:17 | efried | nova meeting now-ish | |
| 13:59:49 | shilpasd | efried: hi | |
| 13:59:53 | efried | hello | |
| 14:03:50 | shilpasd | efried: dansmith: here added one question, https://review.opendev.org/#/c/694462/8/nova/db/sqlalchemy/api.py@2008 | |
| 14:04:16 | shilpasd | help me to understand 'any issues in setting 'volume_backed' field on the existing instances in the upgrade db script?' | |
| 14:05:13 | efried | shilpasd: dansmith is on vacation this week, and I'm not really qualified to discuss this one. It's Chinese New Year around now, so I'm not sure if alex_xu is around either. | |
| 14:05:23 | efried | IOW this may have to wait until next week to close on. | |
| 14:05:50 | shilpasd | efried: ok, thanks | |
| 14:06:13 | shilpasd | will wait for them | |
| 14:12:32 | sean-k-mooney | well that was quick :) | |
| 14:12:54 | shilpasd | :] (y) | |
| 14:13:18 | sean-k-mooney | looking at dansmith's suggestion i tened to agree that offloadign this to sql or adding a new field would be better | |
| 14:16:59 | sean-k-mooney | you could perhaps store the "is_volume_backed" info in the system_metadata table | |
| 14:17:24 | sean-k-mooney | that woudl avoid the need for a schema change but i think option 1 makes the most sense too | |
| 14:17:53 | sean-k-mooney | the only thing is we would have to keep the logic to handel old instance for a few release for FFU | |
| 14:18:06 | sean-k-mooney | but eventurlaly we could drop it | |
| 14:18:22 | shilpasd | sean-k-mooney: thanks for your opinion | |
| 14:18:38 | shilpasd | please clear me last point 'keep the logic to handel old instance for a few release for FFU' | |
| 14:19:39 | sean-k-mooney | we "support" skip level upgrade via Fast Forward upgrdes at lest downstream in OSP/tripleo | |
| 14:20:00 | sean-k-mooney | so if we add the code handel loading old instances and calulating the value | |
| 14:20:14 | sean-k-mooney | we need to keep it for a cople of release before we can remove it | |
| 14:20:21 | sean-k-mooney | and safly assume it will always be set | |
| 14:20:55 | sean-k-mooney | eventually that code for old instance can be removed but not for 2-3 releases | |
| 14:22:22 | shilpasd | for old instances will update 'volume_backed' based on bdms.volume and bdms.boot_index | |
| 14:23:38 | shilpasd | so still not clear why we need to keep code for couple of release, please elaborate further | |
| 14:23:48 | sean-k-mooney | its technically also stored in the request spec for instance created after rocky | |
| 14:24:12 | sean-k-mooney | FFU do not require the compute agents to start during the upgrade | |
| 14:24:35 | sean-k-mooney | so you can go form qeens to train without ever starting the agent untill you get to train | |
| 14:25:01 | sean-k-mooney | so unless you are groin to write an online data migration for this | |
| 14:25:10 | sean-k-mooney | which is not what dansmith was suggesting | |
| 14:25:48 | sean-k-mooney | if you take the approch of fixing it when you load the objects you need to keep that code for a few releases to not force peopel that are doing FFU to stop in ussuri | |
| 14:25:55 | sean-k-mooney | start the agent and have it fix the instnaces | |
| 14:27:05 | sean-k-mooney | so all i was really saying is eventaully we can remove that compatiablity code for old instnace but we will have to wait a releast or 3 | |
| 14:29:33 | shilpasd | ok, got it | |
| 15:09:28 | sean-k-mooney | by the way we can recheck stuff currently right | |
| 15:09:57 | sean-k-mooney | there is not specific gate blockers at the moment out side of the intermitenly failing jobs | |
| 16:03:43 | stephenfin | sean-k-mooney: not that I'm aware of, no | |