| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-23 | |||
| 12:25:25 | kashyap | Alright, fixed the reno. | |
| 12:26:55 | kashyap | gibi: Your patient wait is over :D | |
| 12:27:30 | gibi | done | |
| 12:27:52 | gibi | today something is wrong with my uplink | |
| 12:31:01 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/stein: Mask the token used to allow access to consoles https://review.opendev.org/702181 | |
| 12:31:18 | gibi | elod: fixed your comments in ^^ | |
| 12:37:21 | sean-k-mooney | stephenfin: have you seen issues with nova.tests.functional.test_nova_manage.TestDBArchiveDeletedRowsMultiCell failing out of interest? | |
| 12:37:34 | stephenfin | I haven't paid attention to it, no | |
| 12:38:19 | sean-k-mooney | ok i was wondering if that was the failing db test ye were talking about yesterday | |
| 12:38:22 | sean-k-mooney | i guess not | |
| 12:39:45 | elod | gibi: thx, looking | |
| 12:54:37 | stephenfin | sean-k-mooney: Don't know if I showed this to you before Xmas or not https://github.com/testing-cabal/subunit/pull/40 | |
| 12:55:36 | stephenfin | I don't understand Python's IO model well enough to come up with better, but that fixed things for me for https://review.opendev.org/#/c/682111/ anyway | |
| 12:55:59 | stephenfin | whoops | |
| 12:55:59 | sean-k-mooney | you did not but ill take a look at both | |
| 12:56:12 | stephenfin | https://review.opendev.org/#/c/700522/ | |
| 12:56:49 | sean-k-mooney | why are you importing the print fucntion explcitly | |
| 12:56:57 | sean-k-mooney | are you using py26 lol | |
| 12:57:38 | sean-k-mooney | you can still do that but it was never needed in py27 | |
| 12:58:19 | stephenfin | it definitely is/was :) | |
| 12:58:37 | sean-k-mooney | but ya does that work | |
| 12:58:45 | stephenfin | to use the 'file=foo' thing, anyway | |
| 12:58:58 | sean-k-mooney | oh ya that is python3 only | |
| 12:59:06 | stephenfin | 'zactly | |
| 12:59:20 | sean-k-mooney | but again nova has droped python 2 support so :P | |
| 12:59:36 | stephenfin | so that nova patch just prints a load of junk that overwhelms subunit | |
| 12:59:46 | stephenfin | without my subunit change, it craps out with the subunit parser error | |
| 12:59:55 | stephenfin | with it, it still craps out but with a proper error | |
| 13:00:04 | jawad_axd | Quick question guys: What does cpu allocation 2.0 means ? Is it 2:1? | |
| 13:00:16 | stephenfin | saying the packet is > 4k (I think) | |
| 13:01:06 | jawad_axd | and it is safe to change cpu allocation ratio in running environment? | |
| 13:01:14 | stephenfin | sorry, 4M https://github.com/testing-cabal/subunit/blob/master/python/subunit/v2.py#L202-L208 | |
| 13:01:43 | stephenfin | jawad_axd: yup, 2:1 (20 enabled host CPUs = 40 VCPU inventory) | |
| 13:02:18 | stephenfin | jawad_axd: It should be safe to change so long as you don't lower it to the point that there's less inventory available than you have used | |
| 13:02:56 | stephenfin | I'm not actually sure what would happen then. I assume the periodic task to update placement's inventory would start failing | |
| 13:03:18 | stephenfin | easily tested in a pre-prod environment :) | |
| 13:03:41 | sean-k-mooney | stephenfin: ok well if we have a repoducer that means we have a chance of fixing it | |
| 13:03:57 | sean-k-mooney | we dont need nova at all we can repodcuse this in stestr | |
| 13:04:02 | stephenfin | Exactly. That's pretty damn consistent | |
| 13:04:20 | stephenfin | We should be able to but I couldn't do so when I tried | |
| 13:04:29 | stephenfin | Probably didn't have the correct fixtures configured or something | |
| 13:05:10 | sean-k-mooney | hum ok well i might take a look at this later. and see if i can figure something out | |
| 13:05:33 | sean-k-mooney | if we can create a simpler repoducer that woudl be good if not this works | |
| 13:06:58 | jawad_axd | @stephenfin Thanks. One more thing, In horizon I see cpu's under compute(hypervisor ) tab as 24cpu. So with 2.0 cpu allocation ratio, I should be able to use 24x2=48 vcpus, right? But I am only able to use 24 cpus.What you say about it? | |
| 13:07:42 | stephenfin | I imagine Horizon is pulling that info from the os-hypevisor API which doesn't take overcommit ratios into account | |
| 13:07:53 | sean-k-mooney | i think so too | |
| 13:08:03 | sean-k-mooney | i think horizon is showing the correct value | |
| 13:08:29 | sean-k-mooney | the allocation raitio can be change per host it would not be resonable to have to compute it differnetly per hosts | |
| 13:09:53 | jawad_axd | If horizon is showing the same cpu's on host. Does it mean, overcommitment is not being used/applied ? | |
| 13:10:04 | sean-k-mooney | no | |
| 13:10:23 | sean-k-mooney | over commit is calulated in the scheduler/placment | |
| 13:10:32 | sean-k-mooney | it should not be see in horizon | |
| 13:10:36 | stephenfin | what sean-k-mooney said | |
| 13:10:55 | stephenfin | Horizon might go into negative available values (I'm not sure) but instances will still be scheduled | |
| 13:11:32 | sean-k-mooney | jawad_axd: if you are using a recent version of openstack and look at the placment RP for the host you will see an cpu inventory where the total = the number of cores on the host and the allocation ratio will be 2.0 | |
| 13:11:55 | sean-k-mooney | well on my home system it currently shos 44/24 | |
| 13:12:10 | sean-k-mooney | so it does not go into negitiv but the used can exceed the available | |
| 13:12:19 | jawad_axd | I am just stuck because I can not use more than 24 cpus, while with overcommitment I should be able to use 48. | |
| 13:12:38 | jawad_axd | I am using stein | |
| 13:12:42 | sean-k-mooney | you cannot use more the 24cpu in one guest or in general | |
| 13:13:05 | jawad_axd | In general. | |
| 13:13:09 | sean-k-mooney | did you set the cpu allcoation ratio in the compute node config | |
| 13:13:11 | stephenfin | also, how are you creating the instance? Via the Horizon UI or on the CLI? | |
| 13:13:25 | jawad_axd | # Scheduler | |
| 13:13:25 | jawad_axd | cpu_allocation_ratio = 2.0 | |
| 13:13:36 | sean-k-mooney | ya that does not work in stien | |
| 13:13:43 | jawad_axd | From both, cli and horizon | |
| 13:13:44 | sean-k-mooney | you hage to set it per compute node | |
| 13:13:55 | jawad_axd | ah ok. | |
| 13:14:13 | jawad_axd | how to set per compute node? Can you give me some pointers plz? | |
| 13:14:46 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.cpu_allocation_ratio | |
| 13:15:00 | sean-k-mooney | you just set cpu_allocation_ratio in the default section | |
| 13:15:09 | sean-k-mooney | in the nova.conf | |
| 13:15:35 | sean-k-mooney | alternitivly if you want to manage it via the placement api you can use initial_cpu_allocation_ratio | |
| 13:18:38 | jawad_axd | So cpu_allocation_ratio in nova.conf at each compute node will work for that compute node? | |
| 13:18:58 | sean-k-mooney | correct it should | |
| 13:19:24 | sean-k-mooney | what i belive is happening is that the allcoation raition in the placement inventory is set to 1 | |
| 13:19:43 | sean-k-mooney | and that is causeign the host to be eliminated before you get to the schduler | |
| 13:19:47 | sean-k-mooney | where you set it to 2 | |
| 13:19:58 | sean-k-mooney | you could check that | |
| 13:20:19 | sean-k-mooney | you can do "openstack resource provider list" | |
| 13:20:48 | sean-k-mooney | then do "openstack resource provider inventory show <RP uuid>" | |
| 13:20:57 | rouk | cant use 24 vcore or more per guest? when was this? i have 32vcore guests. | |
| 13:21:51 | sean-k-mooney | you can never have more cores in the guest then are on the host but you can have multple guests sharing cores on the host | |
| 13:22:42 | sean-k-mooney | so on my home server i currently have 44 guest cpus spread over 20 host cpus | |
| 13:22:49 | sean-k-mooney | but i could not boot a 44 core vm | |
| 13:23:51 | jawad_axd | @sean-k-mooney ok | |
| 13:29:35 | jawad_axd | @sean-k-mooney Yeah "openstack resource provider inventory show 04b5aba0-ca20-4e15-881b-eaca3ce73db7 VCPU" show overcommitment 24x2=48. So probably, I was using more than 24cpus for one guest which I cant do as you said. | |
| 13:33:03 | sean-k-mooney | jawad_axd: there are some other constrats as well | |
| 13:33:12 | sean-k-mooney | if you are using cpu pinning you cannot over commit | |
| 13:33:41 | sean-k-mooney | and if you are using a virtual numa toplgoy then within each numa node the cpus cant be over commited | |
| 13:35:01 | jawad_axd | ok my hosts have mostly this architecture, Can you have a look at it plz. http://paste.openstack.org/show/788715/ | |
| 13:36:14 | sean-k-mooney | ya so you have 4 numa nodes and 12 cores per numa node | |
| 13:36:40 | sean-k-mooney | well 12 threads | |
| 13:36:53 | jawad_axd | Does it allow overcommitment ? | |
| 13:37:04 | sean-k-mooney | sure | |
| 13:37:10 | sean-k-mooney | but it depends on the vm | |
| 13:37:28 | sean-k-mooney | if the vm has a numa toplogy it impacts how the vm can be placed | |
| 13:37:30 | jawad_axd | If vms are using cpu host-model, then? | |
| 13:37:37 | sean-k-mooney | no | |