BETA edition rcu errors
#11
(05-14-2021, 08:28 PM)smaeul Wrote: Yes, the frequency of the errors depends greatly on the CPU clock rate. Lower CPU clock seems to be better, though there's no obvious pattern. It also varies greatly for me from run to run; I added random delays to the tool to help compensate for that.

I sent a patch to Linux based on your data (thanks again!): https://lore.kernel.org/linux-sunxi/2021...and.org/T/

If you are able to recompile your kernel, please try with that patch and see if your issues are resolved. If so, feel free to respond to that email with:

Quote:Reported-and-tested-by: Your Name <your@email>

Though you don't have to by any means (if you prefer to stay pseudonymous).

Applied the patch to mobian-5.10 https://gitlab.com/mobian1/devices/sunxi64-linux

Tested the 3 devices I previously reported the errors on. All passed 30 passes (1 hour) without any errors.

I have 2 more devices to test. I am also running it overnight on two of the worst performing devices.
  Reply
#12
@smaeul Thanks for this patch and all the other work you've done on the A64/PinePhone.

I have tested the patched kernel on all 5 of the PinePhone mainboards that were having the rcu errors. All passed without any errors, two boards I ran timer_test for 720 passes (about 24hours) with no errors.

How long do you think it will take for this patch to go live in the main PinePhone distributions? I suspect a lot of people's problems with instability will be fixed by this. I have a 40% failure rate without this patch, 5 out of 12 PinePhone mainboards. 3 boards are really bad, 2 from eBay, I suspect those users gave up on the PinePhone because of the constant crashing.
  Reply
#13
Arch Linux Arm has released linux-pine64-5.12.4-1 kernel which has the patch. Running it now.

-----

Looks like anything based off https://github.com/megous/linux/releases...10519-1214 will have the fix.
  Reply
#14
The results are awesome!  Improved reliability will be great for the PinePhone and all A64-based SBCs, and for our beloved SOM, SOPine.  Great work, @smaeul, and good testing, @8bit!  Thank you both very much! 

I'll submit a patch for U-Boot that brings it to the same "patch level" (i.e. mask width) as the Linux kernel, which should resolve random lockups in U-Boot exeperienced on the PinePhone by myself and a few other forum members.
  Reply
#15
Manjaro has patched U-Boot in testing/unstable branches. https://gitlab.manjaro.org/manjaro-arm/p...c205b71d69

It has fixed the boot hang on the 4 devices that I have that would hang 25% of the time. I have booted two of them more than 100 times without hanging.
  Reply
#16
Arch Linux Arm has updated U-Boot with the patch.
https://github.com/dreemurrs-embedded/Pi...452f731228
  Reply
#17
(05-25-2021, 01:12 PM)8bit Wrote: It has fixed the boot hang on the 4 devices that I have that would hang 25% of the time. I have booted two of them more than 100 times without hanging.

Great results!  Boot hangs were one of the remaining few stability issues with the PinePhone.  Thank you for the testing.
  Reply
#18
Bug reappears in still present. Kernel 5.14.14-2 from Manjaro ARM Phosh.

bug report here: https://gitlab.manjaro.org/manjaro-arm/p.../issues/14
  Reply
#19
(10-27-2021, 08:29 AM)some_pinephone_user Wrote: Bug reappears in still present. Kernel 5.14.14-2 from Manjaro ARM Phosh.

bug report here: https://gitlab.manjaro.org/manjaro-arm/p.../issues/14

Ran the timer_test for 1 hour and did not have any errors. This was on 5.14.14-2-MANJARO-ARM #1 SMP Sun Oct 24 12:31:02 UTC 2021 aarch64 GNU/Linux
This phone would produce errors  every few seconds before the patch.

Also ran on another previously problematic phone running ARCH 5.14.14-1-danctnix #1 SMP PREEMPT Tue Oct 26 16:28:12 UTC 2021 aarch64 GNU/Linux without errors.

It is possible that particular phone has a SOC with a seriously buggy timer. Should run the timer_test as described here https://forum.pine64.org/showthread.php?...9#pid94389
  Reply
#20
I ran the test and got fails inmediately: https://pastebin.com/xbTeksJN

The second ran get the fails inmediately too.
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Severe screen flicker + occasional ghosting | Pinephone Beta edition legowave440 4 313 04-23-2022, 02:04 AM
Last Post: fxc
Question Upgrading a 2020 Community Edition Pinephone? danimations 5 708 03-10-2022, 08:18 AM
Last Post: danimations
  Offer: PinePhone UB Edition in Germany firefox-58 2 1,115 12-11-2021, 11:26 AM
Last Post: Dr. Dreamer
  PinePhone Beta Edition will not boot from MicroSD card Timothy_Ecc 24 10,360 09-25-2021, 09:06 AM
Last Post: 8bit
Sad PnePhone Braveheart Edition - Microphone NOT Working valentin 15 2,066 08-09-2021, 07:43 PM
Last Post: tllim
  Vibrator not working - Manjaro Community Edition Beta3 Gerhard 2 1,830 11-29-2020, 07:52 AM
Last Post: saba
Question No HDMI signal from new Manjaro Community Edition phone dock brianary 13 8,160 11-29-2020, 07:22 AM
Last Post: wibble
  pmOS CE edition screen brightness disappears totally when dock plugged in. pinacolada 0 1,031 11-17-2020, 05:07 PM
Last Post: pinacolada
  Modem UBports edition Celluar not working FOSSagent0 18 11,175 08-15-2020, 02:26 PM
Last Post: bcnaz
Question UBports Edition Flashing eMMC slow speeds FOSSagent0 0 1,125 08-12-2020, 07:03 AM
Last Post: FOSSagent0

Forum Jump:


Users browsing this thread: 1 Guest(s)