Please help me troubleshoot strange network performance
Hi, first post here!

My big project this summer was to build a NAS with a RockPro64 at its heart, making use of its PCIe connector to connect an LSI SAS HBA and a SAS expander to connect much more than 4 drives. On top of that, I was able to install ZFS quite easily and use it as a remote backup server for my main NAS. I don't thinks there's many other SBCs than can handle all of that.

After a lot of reading around here and Github, I only tried installing ayufan's pre-release from April, because it added the neccessary support for the LSI card. I chose the OMV version buster-openmediavault-rockpro64-0.10.12-1184-arm64.img.xz and things have been more or less smooth (my gripes are about OMV more than the RP64).

Now, the problem is that I'm trying to send backups between my servers and I'm hitting a strange 300 KByte/s bottleneck when sending from the remote RP64 to my local proxmox Ryzen server. The same test in the other direction achieves about 30 MByte/s (both locations have 300 Mbit fiber internet connections).

Next I've tried with iperf and a Mac Mini that is in the same location as the RockPro64 and the results stump me again:

Proxmox    22 MBytes/s  ----(cloud)--->    Mac Mini
        10~20 MBytes/s  <--(_______)---   

Proxmox    35 MBytes/s  -----(cloud)--->   RockPro64
          0.37 MBytes/s <---(_______)---   

RockPro64  ~30 MBytes/s  -----LAN--->    Mac Mini
          20~40 MBytes/s <----------

So it seems like the RockPro is capable of sending 30 MByte/s upsteam its network port, just not towards my Proxmox server.
And the router can send me a reasonably fast stream, just not from my RockPro64 Angry

I've tested raw iperf3,  zfs sends through ssh, zfs sends through Wireguard... the 300 Kb/s limit is consistent.

Please help me, what is it that I'm missing? Huh
With mainline kernel I had some problems with TCP connections. By inspecting them with Wireshark I saw that some IP packets were dropped and asked for retransmition likely due to bad checksum. The problems disappeared when I turned off "TCP checksum offload" from the network device:

# ethtool -K eth0 tx off

Possibly Related Threads…
Thread Author Replies Views Last Post
  RPro64s not connecting to home network Luke 10 5,839 06-05-2020, 08:38 AM
Last Post: Luke
  poor HDD performance - ayufan mainline OMV Mentaluproar 0 1,170 11-11-2019, 10:26 PM
Last Post: Mentaluproar
  strange behaviour on newer kernel tuxd3v 3 2,734 10-11-2018, 06:37 PM
Last Post: tuxd3v
  Network boot Drago 1 1,666 09-04-2018, 12:28 PM
Last Post: dukla2000

Forum Jump:

Users browsing this thread: 1 Guest(s)