+- PINE64 (https://forum.pine64.org)
+-- Forum: PinePhone (https://forum.pine64.org/forumdisplay.php?fid=120)
+--- Forum: PinePhone Hardware (https://forum.pine64.org/forumdisplay.php?fid=122)
+--- Thread: BETA edition rcu errors (/showthread.php?tid=13786)
RE: BETA edition rcu errors - 8bit - 05-17-2021
(05-14-2021, 08:28 PM)smaeul Wrote: Yes, the frequency of the errors depends greatly on the CPU clock rate. Lower CPU clock seems to be better, though there's no obvious pattern. It also varies greatly for me from run to run; I added random delays to the tool to help compensate for that.
I sent a patch to Linux based on your data (thanks again!): https://lore.kernel.org/linux-sunxi/20210515021439.55316-1-samuel@sholland.org/T/
If you are able to recompile your kernel, please try with that patch and see if your issues are resolved. If so, feel free to respond to that email with:
Quote:Reported-and-tested-by: Your Name <your@email>
Though you don't have to by any means (if you prefer to stay pseudonymous).
Applied the patch to mobian-5.10 https://gitlab.com/mobian1/devices/sunxi64-linux
Tested the 3 devices I previously reported the errors on. All passed 30 passes (1 hour) without any errors.
I have 2 more devices to test. I am also running it overnight on two of the worst performing devices.
RE: BETA edition rcu errors - 8bit - 05-18-2021
@smaeul Thanks for this patch and all the other work you've done on the A64/PinePhone.
I have tested the patched kernel on all 5 of the PinePhone mainboards that were having the rcu errors. All passed without any errors, two boards I ran timer_test for 720 passes (about 24hours) with no errors.
How long do you think it will take for this patch to go live in the main PinePhone distributions? I suspect a lot of people's problems with instability will be fixed by this. I have a 40% failure rate without this patch, 5 out of 12 PinePhone mainboards. 3 boards are really bad, 2 from eBay, I suspect those users gave up on the PinePhone because of the constant crashing.
RE: BETA edition rcu errors - 8bit - 05-19-2021
Arch Linux Arm has released linux-pine64-5.12.4-1 kernel which has the patch. Running it now.
-----
Looks like anything based off https://github.com/megous/linux/releases/tag/orange-pi-5.12-20210519-1214 will have the fix.
RE: BETA edition rcu errors - dsimic - 05-23-2021
The results are awesome! Improved reliability will be great for the PinePhone and all A64-based SBCs, and for our beloved SOM, SOPine. Great work, @smaeul, and good testing, @8bit! Thank you both very much!
I'll submit a patch for U-Boot that brings it to the same "patch level" (i.e. mask width) as the Linux kernel, which should resolve random lockups in U-Boot exeperienced on the PinePhone by myself and a few other forum members.
RE: BETA edition rcu errors - 8bit - 05-25-2021
Manjaro has patched U-Boot in testing/unstable branches. https://gitlab.manjaro.org/manjaro-arm/packages/core/uboot-pinephone/-/commit/ebdfd9a4124276fdb2d636aaecb861c205b71d69
It has fixed the boot hang on the 4 devices that I have that would hang 25% of the time. I have booted two of them more than 100 times without hanging.
RE: BETA edition rcu errors - 8bit - 05-26-2021
Arch Linux Arm has updated U-Boot with the patch.
https://github.com/dreemurrs-embedded/Pine64-Arch/commit/686f4b4cf48c1ed50fbdfbd3ac0c2f452f731228
RE: BETA edition rcu errors - dsimic - 06-04-2021
(05-25-2021, 01:12 PM)8bit Wrote: It has fixed the boot hang on the 4 devices that I have that would hang 25% of the time. I have booted two of them more than 100 times without hanging.
Great results! Boot hangs were one of the remaining few stability issues with the PinePhone. Thank you for the testing.
RE: BETA edition rcu errors - some_pinephone_user - 10-27-2021
Bug reappears in still present. Kernel 5.14.14-2 from Manjaro ARM Phosh.
bug report here: https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephone/-/issues/14
RE: BETA edition rcu errors - 8bit - 10-27-2021
(10-27-2021, 08:29 AM)some_pinephone_user Wrote: Bug reappears in still present. Kernel 5.14.14-2 from Manjaro ARM Phosh.
bug report here: https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephone/-/issues/14
Ran the timer_test for 1 hour and did not have any errors. This was on 5.14.14-2-MANJARO-ARM #1 SMP Sun Oct 24 12:31:02 UTC 2021 aarch64 GNU/Linux
This phone would produce errors every few seconds before the patch.
Also ran on another previously problematic phone running ARCH 5.14.14-1-danctnix #1 SMP PREEMPT Tue Oct 26 16:28:12 UTC 2021 aarch64 GNU/Linux without errors.
It is possible that particular phone has a SOC with a seriously buggy timer. Should run the timer_test as described here https://forum.pine64.org/showthread.php?tid=13786&pid=94389#pid94389
RE: BETA edition rcu errors - some_pinephone_user - 10-27-2021
I ran the test and got fails inmediately: https://pastebin.com/xbTeksJN
The second ran get the fails inmediately too.