| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-07-08 | |||
| 08:07:47 | bauzas | sar gives me a lot of information | |
| 08:08:07 | bauzas | but this is clear this is just a CPU-bound task, that's it | |
| 08:08:12 | bauzas | not a lot of context switching | |
| 08:08:24 | bauzas | my sys % is high, about 20% | |
| 08:08:30 | bauzas | and user % is about 60% | |
| 08:08:36 | bauzas | so I straced | |
| 08:08:41 | bauzas | and I see lots of polling | |
| 08:08:47 | gibi | good morning | |
| 08:09:31 | Uggla | bauzas, mine is also slow sometime when it is getting hot. I think the cooling system is a bit limited. | |
| 08:09:36 | Uggla | gibi, o/ | |
| 08:09:49 | bauzas | Uggla: good point, I should dig sar's power information | |
| 08:10:12 | gibi | Uggla: give me 20 mins and I'm all yours if neede | |
| 08:10:12 | gibi | d | |
| 08:10:17 | bauzas | and yeah, in general, my fan is yelling | |
| 08:10:44 | bauzas | I suspected that the CPU is stepping down | |
| 08:10:51 | bauzas | hence the perf issues | |
| 08:11:05 | bauzas | but that's a guess | |
| 08:11:06 | Uggla | bauzas, in such case, I can not switch the power balance to performance, it sticks to balanced. | |
| 08:11:24 | bauzas | Uggla: me too, can't modify it | |
| 08:11:46 | bauzas | this is like I was running my laptop unplugged | |
| 08:11:59 | bauzas | which shouldn't be the rule | |
| 08:12:22 | bauzas | if my laptop is plugged, I shall expect max performance and CPU power not stepping down | |
| 08:12:47 | bauzas | but yeah, haven't checked the temperature | |
| 08:12:56 | bauzas | you make me realize a thing | |
| 08:13:13 | bauzas | I think my fan was yelling because the CPU was intensively high | |
| 08:13:17 | bauzas | thought* | |
| 08:13:24 | bauzas | but that could be the contrary | |
| 08:13:40 | bauzas | if the temperature rises, then the fan yells and then the CPU could step down | |
| 08:14:11 | bauzas | all of this seems tied to Fedora power management tools | |
| 08:14:37 | bauzas | but yeah, needs a reproducer to validate | |
| 08:14:54 | bauzas | Uggla: either way, I was having performance trouble with F35 already | |
| 08:15:00 | bauzas | so F36 isn't a regression | |
| 08:15:28 | Uggla | bauzas, I think it is what happened in my case. And that's dramatically long when you run the openstack tests suite. | |
| 08:16:10 | bauzas | point is, the TB3 system doesn't help | |
| 08:16:20 | bauzas | previously with my other laptops, those were docked | |
| 08:16:33 | bauzas | so the fan was having room for air | |
| 08:16:52 | bauzas | now, with the tb3 plug, the laptop is just sit down | |
| 08:16:54 | Uggla | clearly | |
| 08:17:42 | bauzas | will continue to dig into it | |
| 08:17:50 | bauzas | there are actions I could do | |
| 08:17:57 | bauzas | and I'd like to verify | |
| 08:18:25 | bauzas | maybe the cooling system is sufficient and this is only Fedora which is too much conservative in terms of power consumption | |
| 08:18:40 | bauzas | having the fan burning isn't a problem | |
| 08:18:52 | bauzas | that's the CPU stepping down which is an issue | |
| 08:19:32 | Uggla | yep I think so as well. | |
| 08:19:46 | gibi | I also have perf issues with google meet (on T14, debian sid, xorg, i3, chrome). During our calls sometimes chrome starts to hogg the CPU and the audio and video starts lagging, and clipping. I haven't figured out yet what triggers the CPU hogging | |
| 08:20:28 | gibi | but as CPU get hot, thermal throttling kicks in and worsen the situation | |
| 08:21:10 | Uggla | gibi, welcome to the club :) | |
| 08:21:20 | gibi | then after 1 minute chrome stops hogging the CPU and after an additional minute google meet audio/video recovers | |
| 08:21:47 | gibi | I suspect that in my case it is chrome that does something crazy | |
| 08:23:40 | Uggla | bauzas, maybe we should have a look to firmware as well. | |
| 08:24:22 | Uggla | *has | |
| 08:25:13 | Uggla | gibi, have you a moment to look at my issue ? | |
| 08:25:20 | gibi | yes, I'm ready now | |
| 08:25:47 | gibi | lets test some google meet ;) | |
| 08:25:53 | Uggla | yep | |
| 08:25:54 | Uggla | :) | |
| 08:34:22 | bauzas | ok, I tuned sar to report my CPU clock freq | |
| 08:34:43 | bauzas | we'll see, I have to join auniyal_'s meeting now :) | |
| 08:39:19 | bauzas | hah ! | |
| 08:39:44 | bauzas | my clock freq turns down to 800MHz while the temperature on the devices is only about 50°C | |
| 08:39:44 | bauzas | my clock freq turns down to 800MHz while the temperature on the devices is only about 50°C | |
| 08:40:05 | gibi | that sounds bad | |
| 08:40:19 | bauzas | when I playing gmeet I mean | |
| 08:40:21 | gibi | mine can sustain 1.8GHz while hot | |
| 08:41:12 | bauzas | sar reports the frequency constantly flipping between 866MHz and 1.5GHz | |
| 08:41:19 | bauzas | every second | |
| 08:42:48 | bauzas | yeah, found a direct correlation | |
| 08:42:50 | bauzas | huzzah | |
| 08:43:05 | bauzas | between the CPU clock freq and the CPU usage | |
| 08:47:29 | bauzas | https://paste.opendev.org/show/bHWo1z8ElaEDqW2b95fZ/ | |
| 08:51:35 | bauzas | [sbauza@sbauza ~]$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor | |
| 08:51:35 | bauzas | powersave | |
| 08:51:38 | bauzas | hmmmpfff | |
| 08:56:45 | kashyap | bauzas: What laptop is yours? | |
| 09:04:29 | bauzas | 11:02:30 CPU MHz | |
| 09:04:29 | bauzas | 11:02:31 all 410,34 | |
| 09:04:33 | bauzas | t14s | |
| 09:05:21 | kashyap | I see | |
| 09:30:47 | Uggla | bauzas, maybe you can run sudo fwupdmgr get-devices it will give you fw revision on your laptop. So we could compare with mine and gibi's. | |
| 09:31:18 | bauzas | on a meeting with auniyal_ | |
| 09:31:23 | bauzas | will reply later | |
| 09:31:43 | bauzas | but I think this is a the CPU governor's fault | |
| 09:33:12 | gibi | I have a ThinkPad T14s Gen 1 | |
| 09:33:48 | gibi | hm, the powersave governor could be a problem | |
| 09:34:01 | Uggla | bauzas from release note latest system relase note: • Updated includes thermal optimization. | |
| 09:35:55 | Uggla | clearly mine is not at the latest version 0.1.15 vs 0.1.21 | |
| 09:38:52 | Uggla | so bauzas has you have received this laptop first maybe maybe your fws are really "old". | |
| 09:39:55 | Uggla | My current version: https://paste.openstack.org/show/b9cnjsnSNOqzjw52AA8L/ | |
| 09:47:45 | gibi | this is mine https://paste.opendev.org/show/b48RVDFNHGRaeHUq2qMg/ | |
| 10:10:11 | sean-k-mooney1 | bauzas: powersave keep the cpu at the minium frequency | |
| 10:10:42 | sean-k-mooney | you should set it to the hardware pstate dirver or use ondemand | |
| 10:10:56 | sean-k-mooney | also i recommend not using tuned | |
| 10:11:14 | sean-k-mooney | i have had severl issue with it in the past so i dont trust it anymore | |
| 10:12:11 | sean-k-mooney | by issue i mean putting one or both ports in a bond to sleep or disabling my mouse/keyboard randomly | |
| 10:14:55 | bauzas | sean-k-mooney1: yup, was considering to use ondemand | |
| 10:15:14 | bauzas | sean-k-mooney1: but I'd like a governor different between being plugged or not | |
| 10:15:24 | bauzas | I thought Fedora was managing it but apparently not | |
| 10:15:46 | sean-k-mooney | bauzas: i used to use udev rules to do that | |
| 10:15:57 | bauzas | yup, that's what I thought I need to do | |