Segfault with GCC in Armbian [Solved].

[ejolson]

I have two 4GB Rock64 boards identically configured.  Each has

• The official 5V 3A power supply.
  
• The official heat sink.
  
• A 32GB SanDisk A1-class SD card.
  
• An Eluteng USB to SATA bridge.
  
• A Sandisk 240GB SSD.
  
• Armbian Focal Server installed for Rock64.
  
• A wired gigabit network connection to same switch.
  
• No monitor attached.
  

I'm trying to compile version 10.2 of gcc.  On one of them the full build completes without problem; on the other I get a segmentation fault or illegal instruction error at random places in the build.

I'm probably past the warranty period because these systems sat on a shelf for a long time after being received due to shelter-at-home rules related to the ongoing epidemic.  If it is possible to exchange the one with intermittent failures, I'd be delighted.  Otherwise, I'm looking for a way to reliably use it.

I would be happy with a solution that achieved reliable operation by

• disabling one of the four CPU cores.
  
• disabling up to 1GB of RAM.
  
• reducing clock speed by up to 20 percent.
  

So far I've tried disabling cores 1 and 3 and using only cores 0 and 2, but that didn't help.  I've tried reducing clock speed to 1GHz but that didn't help.  I've tried cooling with a fan (separately powered) and that didn't help.  Disabling cores 0 and 2 and using only cores 1 and 3 seems a little more promising and testing is still under way.

I haven't tried disabling memory and am not actually sure the way to do this.  Is there an option I can pass to Linux to tell it to ignore certain parts of RAM that might be defective?  What would the suitable memory ranges be for a Rock64 board?

[ejolson]

I forgot to mention that I also tried unplugging the USB to SATA bridge and doing the build using only the SD card and that didn't help either.

I just occurred to me that I could swap the power supplies to see if that makes a difference. I'll be trying that and reporting back. Any other ideas or options would be greatly appreciated.

Is there a way to increase slightly the voltage to memory or the CPU?

[Rocklobster]

I forgot to mention that I also tried unplugging the USB to SATA bridge and doing the build using only the SD card and that didn't help either.

I just occurred to me that I could swap the power supplies to see if that makes a difference. I'll be trying that and reporting back. Any other ideas or options would be greatly appreciated.

Is there a way to increase slightly the voltage to memory or the CPU?

What I would suggest is using an Armbian build on both to see if they boot. I find Armbian is a defacto test when it comes to the Rock64. I have six boards and all six boot with it.

[t4_4t]

> I get a segmentation fault or illegal instruction error at random places in the build.

The memory system seems to have less margin.
See the post below.
> https://forum.pine64.org/showthread.php?tid=7387

I own four boards, one of which showed the same symptoms,
So that one had to use a lower memory clock. (800MHz -> 666MHz)

[tomarm]

Known issue on the rock64 v2.   Only distro that dont segfault during compiling is ayufans ubu from his repo.   Manjaro is the distro I'd like to see it fixed on.   Strit says his v3 has no such problem.

[ejolson]

I forgot to mention that I also tried unplugging the USB to SATA bridge and doing the build using only the SD card and that didn't help either.

I just occurred to me that I could swap the power supplies to see if that makes a difference.  I'll be trying that and reporting back.  Any other ideas or options would be greatly appreciated.

Is there a way to increase slightly the voltage to memory or the CPU?

What I would suggest is using an Armbian build on both to see if they boot. I find Armbian is a defacto test when it comes to the Rock64. I have six boards and all six boot with it.

I'm currently running the Armbian version of Focal Server from Aug 19, 2020 with the 5.7.15 kernel. Both systems have exactly the same configuration, but one is getting segmentation faults and kernel oops. Reading the other replies and linked posts makes me think the memory has been overclocked far beyond reliability and patching the image to initialize the clocks at 333 MHz will likely fix things. I'll be giving this a try shortly.

Are the offsets in the image the same for all distributions?

[ejolson]

> I get a segmentation fault or illegal instruction error at random places in the build.

The memory system seems to have less margin.
See the post below.
> https://forum.pine64.org/showthread.php?tid=7387

I own four boards, one of which showed the same symptoms,
So that one had to use a lower memory clock. (800MHz -> 666MHz)

After reading that thread, this seems exactly what I want to try. I'm a bit lost, however, how to modify uboot to use the 333 MHz initialization. In that thread, it looks like people are patching their boot images with magic offsets. How do I find the offset for my image? Is there a tool for doing this? I've been looking into building uboot myself. It seems Armbian uses a legacy version that has many patches. So again, how did you change the memory clock?

[ejolson]

I'm a bit lost, however, how to modify uboot to use the 333 MHz initialization.

It seems patching the SD card was not very difficult.  I downloaded

rk3328_ddr_333MHz_v1.16.bin
rk3328_miniloader_v2.46.bin

Then I followed the instructions on

http://opensource.rock-chips.com/wiki_Boot_option

typed as root

Code:

# mkimage -n rk3328 -T rksd -d rk3328_ddr_333MHz_v1.16.bin idbloader16.img
# cat rk3328_miniloader_v2.46.bin >>idbloader16.img
# dd if=idbloader16.img of=/dev/mmcblk0 seek=64 conv=notrunc
# sync

and then rebooted.

There is a parallel but slightly less helpful thread on the Armbian forum at

https://forum.armbian.com/topic/15082-ro...frequency/

I know it's not so good to run two parallel threads on different boards, but I thought I might need some specific Armbian help with this.  I'll be updating this thread soon with a performance comparison and whether the segmentation faults and kernel oops are gone.  Thanks everyone for the help!

---

I'll be updating this thread soon with a performance comparison and whether the segmentation faults and kernel oops are gone.

Here is a performance comparison between the original 786MHz memory clock and the reduced 333MHz clock when running John McCalpin's stream memory bandwidth benchmark.



The performance reduction for the scale operation was measured to be about 2.26 times which is slightly less than the expected 786/333=2.36 factor loss expected by just dividing out the clocks.

In real world applications much of this performance reduction might be mitigated by cache memory; however, it would seem having a 666MHz setting might be a better compromise between super slow and super unstable.  Does anyone know where something like

rk3328_ddr_666MHz_v1.16.bin

might be found or how to make one?

[ejolson]

This is a followup post to to confirm that the Rock64 board that was experiencing segmentation faults with the default 786 MHz memory clock on Armbian is now stable with the uboot-initialized 333 MHz setting. On a selection of computational benchmarks performance is about 77 percent of the original setting; for compilations with gcc more than 80 percent of the original performance is retained with the added advantage that the system doesn't crash and the compiler doesn't create segmentation faults. I understand, but did not verify, that setting the memory clocks to 333 MHz has a noticeable affect on video playback.

As far as I'm concerned this problem is solved and I'll try to mark it as such. Although one out of the two Rock64 single-board computers I have required this configuration change, based on posts appearing on this forum, my suspicion is a much less than 50 percent--perhaps closer to 10 or 20 percent--of systems in use actually exhibit this problem. Even so, I would favor a note on the wiki support page for the Rock64 explaining how to work around system instability by reducing the DDR RAM memory clock.

[ejolson]

I found the file

rk3328_ddr_400MHz_v1.16.bin

at https://github.com/rockchip-linux/rkbin/...r/bin/rk33 which mitigates the performance loss a little bit compared to the 333 MHz setting from before. Stability still seems fine.