| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-07-08 | |||
| 10:24:01 | sean-k-mooney | oh i tought you did | |
| 10:24:19 | sean-k-mooney | ignore the noise then | |
| 10:24:35 | bauzas | [sbauza@sbauza ~]$ rpm -qa | grep tuned | |
| 10:24:35 | bauzas | [sbauza@sbauza ~]$ | |
| 10:24:57 | bauzas | I'll continue on it this afternoon | |
| 10:25:06 | bauzas | looks like an Hack&Hustle session | |
| 10:26:31 | sean-k-mooney | ya maybe | |
| 10:26:46 | sean-k-mooney | let us know if you figure it out either way | |
| 10:27:06 | bauzas | my sar figures are explicit | |
| 10:27:13 | bauzas | my temperature was still fine | |
| 10:27:31 | bauzas | but I was struggling with CPU clock freq at 400Mhz | |
| 10:27:44 | bauzas | and my audio was then choppy as the load was spiking up to 8.0 | |
| 10:27:54 | bauzas | (8 CPUs, which is normal) | |
| 10:28:21 | bauzas | when the CPU clock was about 1500MHz, then the audio was better (and the load down to 4.0) | |
| 10:28:42 | bauzas | f**** governor | |
| 10:28:53 | sean-k-mooney | wisely or not im using tlp to manage my power https://paste.opendev.org/show/b01fBUjtb7EA4fQDmvuM/ | |
| 10:29:22 | sean-k-mooney | https://linrunner.de/tlp/index.html | |
| 10:30:24 | bauzas | power-profiles-daemon.service loaded active running Power Profiles daemon | |
| 10:31:01 | sean-k-mooney | ya that is one of the things you have to remove if you are using tlp | |
| 10:31:25 | sean-k-mooney | just notifced im actully using schedutil for my govoner | |
| 10:31:40 | sean-k-mooney | if you are not using the intel pstate driver thats actully the better option vs ondemand | |
| 10:31:48 | bauzas | Conflicts=tlp.service shutdown.target auto-cpufreq.service system76-power.service tuned.service | |
| 10:31:56 | bauzas | huhuh | |
| 10:32:04 | bauzas | I'll have to dig into it | |
| 10:32:09 | bauzas | but... lunch | |
| 10:32:11 | sean-k-mooney | yep its one of the ones i tried | |
| 10:32:37 | sean-k-mooney | it didnt work well for me when i use the power-profiles-daemon | |
| 10:33:44 | sean-k-mooney | i think my issue was mainly fan noise and it was not really goign down form max cpu | |
| 10:34:15 | sean-k-mooney | like idling on ac was full non turbo/bost clock speeed | |
| 10:40:15 | opendevreview | Merged openstack/python-novaclient master: Imported Translations from Zanata https://review.opendev.org/c/openstack/python-novaclient/+/848804 | |
| 12:01:55 | bauzas | found the guilty https://fedoraproject.org/wiki/Changes/Power_Profiles_Daemon | |
| 12:02:13 | bauzas | "The "performance" mode is only available on select systems and is implemented by different "drivers" based on the system or systems it targets. " | |
| 12:10:21 | bauzas | that's maybe me but I thought we were having different power management profiles whether the laptop was docked or not, previously | |
| 12:10:32 | bauzas | (in Gnome, I mean) | |
| 12:20:16 | bauzas | https://forums.lenovo.com/t5/Other-Linux-Discussions/Lenovo-L14-Gen1-AMD-Lap-Mode-Throttling/m-p/5154197 | |
| 12:20:51 | bauzas | but my surface is steady... | |
| 12:25:21 | sean-k-mooney | bauzas: so ya i would jsut remove that and go with one of the many other tools | |
| 12:25:35 | bauzas | checked, I'm not in lap mode | |
| 12:25:52 | bauzas | but there are a ton of issues with PPD with Thinkpads https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues | |
| 12:26:40 | bauzas | and that's the issue I hit when changing the PPD profile to 'perf' https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/72 | |
| 12:28:32 | bauzas | sean-k-mooney: any power management tool you'd recommend that could set the power based on charging vs. discharging ? | |
| 12:28:59 | bauzas | powersave seems reasonable to me if I'm running on battery | |
| 12:29:20 | bauzas | but I don't want to clockdown if I'm plugged | |
| 12:30:58 | opendevreview | Merged openstack/nova master: Retry attachment delete API call for 504 Gateway Timeout https://review.opendev.org/c/openstack/nova/+/845543 | |
| 12:33:15 | sean-k-mooney | bauzas: i use tlp | |
| 12:33:33 | sean-k-mooney | i dont actully use power save on battery but i configure it to limit the max frequency | |
| 12:33:55 | sean-k-mooney | but you can certenly set the profile to power save too if you prefer | |
| 12:34:23 | sean-k-mooney | https://paste.opendev.org/show/bPVClbF8MnuFjrhBE2z3/ | |
| 12:34:27 | sean-k-mooney | that is my tlp config | |
| 12:34:54 | sean-k-mooney | bauzas: so you could change line 6 two powersaver | |
| 12:35:03 | sean-k-mooney | or liek i do on line 11 | |
| 12:35:16 | sean-k-mooney | limite the cpu frequency | |
| 12:36:17 | sean-k-mooney | bauzas: tlp also has a tlp-stats tool which is kind of nice | |
| 12:37:22 | sean-k-mooney | you can see that i still have 90% capasity on my battery after 4 years | |
| 12:37:25 | sean-k-mooney | https://paste.opendev.org/show/bNtQTQ0T2Tx4Bd2nFyLu/ | |
| 12:37:37 | sean-k-mooney | partly becasue i only let it chagne to 80% | |
| 12:38:39 | sean-k-mooney | so i went with tlp since i coudl use it to do more then save power it can also extend battry life by contoling change and i cloud have a little more contol over all | |
| 12:39:37 | sean-k-mooney | https://termbin.com/ic06r is the full set of stats it has | |
| 12:39:47 | sean-k-mooney | and you can twek many of the paramters if you care about it | |
| 12:49:55 | opendevreview | Amit Uniyal proposed openstack/nova master: Adds check, if admin has set compute service down https://review.opendev.org/c/openstack/nova/+/848886 | |
| 12:49:56 | opendevreview | Amit Uniyal proposed openstack/nova master: add regression test case for bug 1978983 https://review.opendev.org/c/openstack/nova/+/849104 | |
| 12:50:58 | bauzas | ok, eventually fixed my issue, upgraded the firware | |
| 12:51:02 | gibi | sean-k-mooney: I'm wondering that we reintroduced a an issue by reenabling greendns. See my fresh bugreport https://bugs.launchpad.net/nova/+bug/1981080 | |
| 12:51:15 | bauzas | when the lid is closed, you can't upgrade your firmware, bizarre but ok | |
| 12:51:28 | bauzas | so this one was PEBKAC | |
| 12:51:57 | bauzas | I'll need to use to change the profile based on the docking info | |
| 12:52:04 | bauzas | need to see* | |
| 13:11:00 | sean-k-mooney | gibi: i was still getting that waring before that | |
| 13:11:20 | gibi | hm, I don't get it if I turn off greendns | |
| 13:12:00 | sean-k-mooney | we coudl perhaps disable greendns in our func tests but i dont think a revert would be correct | |
| 13:12:28 | sean-k-mooney | we shoudl not be doing dns lookup in the func tests | |
| 13:13:36 | gibi | this is not about func test, there it is probably cosmetic. But I wonder if we reintroduced https://bugs.launchpad.net/nova/+bug/1808951 | |
| 13:13:57 | gibi | as the fix for that was to blacklist urllib3 | |
| 13:14:49 | gibi | now we see that urllib3 is imported before monkey patching | |
| 13:14:56 | gibi | so we might retrigger https://bugs.launchpad.net/nova/+bug/1808951 | |
| 13:15:08 | sean-k-mooney | well we now use eventliet directly in the nova-api | |
| 13:15:25 | sean-k-mooney | for scater gather so we cant skip monky patching in general there anymore | |
| 13:16:03 | gibi | I don't follow you how that is related | |
| 13:16:21 | sean-k-mooney | we know that disabling greendns can lead to the nova-api and other serivce had locking up on dns quieres | |
| 13:16:33 | sean-k-mooney | so if we revert the other patch we woudl break things | |
| 13:17:06 | gibi | OK, so if we disable greendns we break thing, but now that we enabled it we might break other things | |
| 13:17:29 | sean-k-mooney | we might but i dont think we will | |
| 13:17:58 | sean-k-mooney | have you found a code path where urllib3 is imported before monkeypatching? | |
| 13:18:10 | gibi | yes, it is the dns code in eventlet that imports it | |
| 13:18:41 | sean-k-mooney | so https://github.com/openstack/nova/blob/90c0c687a487601e009c72f60c88be92f6a55264/nova/monkey_patch.py#L30= | |
| 13:18:47 | sean-k-mooney | imports urllib3? | |
| 13:18:54 | gibi | indireclt yes | |
| 13:19:00 | gibi | indirectly | |
| 13:19:06 | sean-k-mooney | that sounds like an eventlet bug that we canot fix then | |
| 13:19:22 | sean-k-mooney | since we cant monkeypatch until after we import eventlet | |
| 13:20:27 | sean-k-mooney | so ya maybe we need to remove urllib3 form the problem list if eventlet have fixed the issue | |
| 13:20:34 | sean-k-mooney | if not then we shoudl file an eventlet bug | |
| 13:20:50 | sean-k-mooney | unless from nova import debugger | |
| 13:20:56 | sean-k-mooney | is pulling it in instead fo eventlet | |
| 13:21:20 | sean-k-mooney | not that i can see | |
| 13:21:24 | sean-k-mooney | https://github.com/openstack/nova/blob/90c0c687a487601e009c72f60c88be92f6a55264/nova/debugger.py | |
| 13:25:34 | gibi | hm I assuming that deployers had some workaround for our disable greendns as we had that disable for a long time | |
| 13:27:05 | gibi | so we might fixed a known and worked around problem with but re-introduced a break somewhere else. Still I feel like reverting the greendns patch would be safer while we figure out is urllib3 is safe | |
| 13:29:21 | gibi | anyhow updated the bug report linking to this discussion as I have now bandwidth to dig deeper into this now | |
| 13:32:09 | Uggla | gibi, sorry for that.... Uggla is a black cat that attract bug. | |