Slow/dead ethernet on (non-Ayufan) Linux Mainline (fixed)
#1
Hello,

I am running vanilla Arch Linux with the stock mainline kernel on my RockPro64 (If you are interested in how to install, just check the other threads).

However, for a while now the ethernet on the RockPro was as good as dead, with network speeds reaching about 50-80Kbit/s!!!

As I am using my RockPro as a home cloud server, you can imagine how frustrating this is for me. This is so slow that even system upgrades fail due to timeouts on downloading the upgrade packages. So I used the free time now to dig into this issue, especially as I have not read anything of this behaviour from the other mainline users in this forum.

The first big difference between me and the other mainline users seem to be that while I am using the mainline Kernel from Arch, others on the Ayufan Images use the kernel compiled from Ayufan's mainline repo, where he also implements some patches specific to the Rockchip devices.

After a while I found a mail from him on the Linux Kernel Mailing Lists,

Thread: https://lkml.org/lkml/2019/4/1/1382

describing my problem as a bug in some rockchip boards where TX checksumming fails for ethernet packets larger than 1498 Bytes due to incorrect buffer utilization. This would also explain why I am still having a connection to my RockPro through SSH and why the DNS server on the RockPro still serves my home network, but updates fail: An ethernet packet can be as big as 65536 Bytes(!), so while SSH and DNS usually stay below a packet size of 1498B due to the small amounts of data they need to transmit, any file download will be split in packets as big as possible, breaking the checksums.

I could nicely verify this behavoiur with Iperf3 on my board: Iperf3 sends 128KB packets as default, reaching the expected 0-50Kbit/s ethernet speed, but when told to send packets of size 1490B only, ethernet throughput suddenly jumps up to gigabit speeds.

So, how to solve this?
It seems that the discussion turns around if this is just a board related issue or something in the SoC, as the problem seems to be in the utilization of the TX FIFO buffers by Linux. As it looks like they are experimenting with the burst lengths and a workaround of using software checksumming instead of offloading the checksumming.

So, I just took the Arch Linux PKGBUILD for the linux-aarch64 stock kernel and tweaked the device tree for the RockPro64 to include the lines

Code:
snps,no-pbl-x8;
snps,txpbl = <0x20>;

in the gmac node. Of course you could also just go to /boot/dts/rockchip, decompile the device tree with dtc, change the lines at the appropriate position and recompile with dtc to save the fuss of having to unnecessarily recompile the whole kernel.

According to the mailing list, just setting
Code:
snps,txpbl = <0x4>;
also work fine.

Changes discussed in the mailing list thread can be found in the commits designated for Linux 5.7, so lets hope this fix does not have to be applied manually for every kernel upgrade in the future.

So, hopefully this thread will become obsolete in a couple of months Big Grin
#2
This sounds exactly like the issue I've been having, which I haven't had a chance to investigate yet (Running NixOS in my case, but same kernel sources). I'll try patching the dts and see if that fixes it. Cheers!


Possibly Related Threads…
Thread Author Replies Views Last Post
  irradium (based on crux linux) RockPro64 riscv64, aarch64 mara 1 345 03-25-2024, 12:12 PM
Last Post: mara
Exclamation Ethernet regression on Linux Kernel 6.5.4? Deathcrow 3 742 09-22-2023, 04:27 AM
Last Post: diederik
  Vanilla mainline Debian 11 (Bullseye) on the RockPro64 Pete Tandy 22 16,829 08-16-2023, 01:34 AM
Last Post: varac
  Installing CH431SER on Ayufan 0.9.14: gitlab-ci-linux-build-159 Thisone 4 1,185 07-14-2023, 04:22 AM
Last Post: hunderteins
  GPIO on Ayufan 0.9.14 Build Thisone 7 1,939 04-23-2023, 01:32 PM
Last Post: diederik
  Linux laptop does not detect the board when plugged in via USB soupy 1 3,920 04-13-2023, 03:01 AM
Last Post: Reynold Grady
  Rockpro64 Dead on arrival? quixoticgeek 1 880 03-12-2023, 06:55 PM
Last Post: quixoticgeek
  RockPro64 linux console video mode callegar 0 834 09-06-2022, 02:32 PM
Last Post: callegar
Brick Maintained Linux booting from eMMC ootoovak 10 7,958 04-30-2022, 03:57 PM
Last Post: TRS-80
  How I Got Fedora Linux to Boot From eMMC (or microSD, for that matter) whitecat23 4 4,105 01-03-2022, 10:32 AM
Last Post: whitecat23

Forum Jump:


Users browsing this thread: 1 Guest(s)