Rock64 4GB poor USB3 HDD performance

[KernelPanic]

I've received my Rock64 4GB a few weeks back but I'm having some trouble getting the external hard drive to perform as it should.

I really hope anyone here has some idea's on how to get performance up to spec because I'm about to pull my hair out with this issue. The main purpose of the Rock64 was a replacement for my RasbPI3 as a NAS. Since the Rock64 has Gigabit ethernet and USB3 it seemed like a good fit but so far it's been dissapointing.  (It's rock stable though )

The drive I'm a Seagate STEA4000400 USB3 (powered by wall adapter) which performs great on my Windows rig. Transferring files from the same server as I'm using to test with the Rock64 gives me a stable 110MB/s up/down.

I'm currently running bionic-minimal-rock64-0.6.31-209-arm64, but I've also tried xenial-minimal-rock64-0.5.15-136-arm64 (the current stable release) and but had similar results.

Right now I'm only get around 20MB/s write and about 40MB/s read on the drive when using RSYNC or SAMBA, which seems way too low.

I've tried Rsync (SSH), then after reading SSH might be the culprit for poor speed changed to NFS, then when this didn't help I tried the Rsync protocol (no notable change in performance).  Then said screw it and tried transferring files via samba, same crap performance. I just tried to do a local rysnc moving files around on the drive, around 18MB/s.

At first the drive was formatted as EXT4, I've formatted it to NTFS now and still get the same performance. (I didn't expect any improvement, but this gave me the change to test the drive on my windows machine, which doesn't have the problem).

Here's some more debugging I've tried to do to narrow down the cause.. (all done with governor @ performance on 4.4.120-rockchip-ayufan-209)

Iperf with Rock64 as client: 

Code:

sudo iperf3 -c 192.168.2.10
\Connecting to host 192.168.2.10, port 5201
[  4] local 192.168.2.101 port 52794 connected to 192.168.2.10 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  96.3 MBytes   805 Mbits/sec    0    946 KBytes
[  4]   1.00-2.00   sec   100 MBytes   839 Mbits/sec    0   1.04 MBytes
[  4]   2.00-3.00   sec   100 MBytes   839 Mbits/sec    0   1.25 MBytes
[  4]   3.00-4.00   sec   100 MBytes   839 Mbits/sec    0   1.25 MBytes
[  4]   4.00-5.00   sec   100 MBytes   840 Mbits/sec    0   1.90 MBytes
[  4]   5.00-6.01   sec   101 MBytes   845 Mbits/sec    0   1.90 MBytes
[  4]   6.01-7.00   sec   100 MBytes   840 Mbits/sec    0   1.90 MBytes
[  4]   7.00-8.00   sec   100 MBytes   842 Mbits/sec    0   1.90 MBytes
[  4]   8.00-9.01   sec   101 MBytes   845 Mbits/sec    0   1.93 MBytes
[  4]   9.01-10.00  sec   100 MBytes   840 Mbits/sec    0   1.93 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   999 MBytes   838 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   995 MBytes   834 Mbits/sec                  receiver

Rock64 as server:

Code:

sudo iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.2.10, port 48590
[  5] local 192.168.2.101 port 5201 connected to 192.168.2.10 port 48592
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   108 MBytes   907 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   941 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   940 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   940 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   941 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   941 Mbits/sec
[  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   941 Mbits/sec
[  5]  10.00-10.03  sec  3.43 MBytes   929 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.03  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.03  sec  1.09 GBytes   937 Mbits/sec                  receiver

As client is a bit slower, but acceptable (if anyone knows why I'd gladly hear it, firewall is disabled)

iozone test:

Code:

iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
       Iozone: Performance Test of File I/O
               Version $Revision: 3.429 $
               Compiled for 64 bit mode.
               Build: linux

       Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                    Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                    Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                    Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                    Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                    Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                    Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                    Vangel Bojaxhi, Ben England, Vikentsi Lapa.

       Run began: Sun Apr  8 19:54:02 2018

       Include fsync in write timing
       O_DIRECT feature enabled
       Auto Mode
       File size set to 102400 kB
       Record Size 4 kB
       Record Size 16 kB
       Record Size 512 kB
       Record Size 1024 kB
       Record Size 16384 kB
       Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
       Output is in kBytes/sec
       Time Resolution = 0.000001 seconds.
       Processor cache size set to 1024 kBytes.
       Processor cache line size set to 32 bytes.
       File stride size set to 17 * record size.
                                                             random    random     bkwd    record    stride
             kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
         102400       4    22657    26762    91373    92627    90584    31170
         102400      16    53854    65291   261840   273802   270460    65326
         102400     512    88260    91791   308807   324209   361771    94020
         102400    1024    85033    92685   356563   368805   368856    92340
         102400   16384    91176    96230   350895   369697   370204    97167

Not sure if these are good results

Test with DD:

Code:

/mnt/sg1/test$ dd if=/dev/zero of=./largefile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 29.5178 s, 36.4 MB/s

Cleared cache with

Code:

sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"

Then tested read, seems fine:

Code:

/mnt/sg1/test$ dd if=./largefile of=/dev/null bs=4k
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.97084 s, 154 MB/s

This is how the drive is mounted in fstab: (ext4 I had some bad perf)

Code:

/dev/disk/by-id/ata-ST4000DM004-2CV104_ZFN0B7VM-part1  /mnt/sg1     ntfs-3g rw, big_writes, defaults,nofail 0 0

Hope someone one of the smart people on here can help me figure this one out.
Thanks in advance!

[Luke]

Hi, could you please download OMV (old, stable) set it up and let us know if you still experience the slow performance?

Thanks.

[KernelPanic]

I've tried the image you provided but am getting similar results, still getting below 20MB/s when uploading files to the Rock64

[KernelPanic]

Tested with a different USB3 device (Sandisk USB stick) which also managed reads above 100MB/s on my windows machine.
Same bad performance on the Rock64 transferring files to my desktop via samba.

I've noticed that when I try download the file a second time speedgoes up too 100MB/s (which matches the iperf3 results), but this is probably because linux cached the file?

[rontant]

Through Samba, I got on average 70MB/s when copying from Win 10 to Rock64 with conventional HDD.


Try putting in "noatime" option in your /etc/fstab. With Linux, if you can, try ext4 format instead of ntfs.

Here is my fstab:

Code:

root@rock64:~# cat /etc/fstab
LABEL=boot /boot/efi vfat defaults 0 1
UUID="c56c18b8-5004-4ea6-ad62-dc62c8251c9x" /ftp/public  ext4   nofail,noatime 0 2
UUID="ca764a03-db3d-49bc-b034-b05abc755ddx" /ftp/public3 ext4  nofail,noatime 0 0

By the way, I am using ayufan's Bionic

Code:

root@rock64:~# uname -a
Linux rock64 4.4.120-rockchip-ayufan-209 #1 SMP Mon Apr 2 16:05:07 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

[thewonderer]

I have the same problem with mine. I did read that SAMBA can be the issue. Plenty of articles on the NET to try and assist, but still stuck around 40MB/s for my HDD. I think people have said FTP is very quick, so will probably test FTP tonight to see if that's the case because running USB benchmarks on my rock64 show the speed maxes out.

The whole reason for me buying the rock64 was to try and get better copy speed across my LAN compared to my banana pi ultra m2.

Will try Bionic soon.

[rontant]

I have the same problem with mine. I did read that SAMBA can be the issue. Plenty of articles on the NET to try and assist, but still stuck around 40MB/s for my HDD.   I think people have said FTP is very quick, so will probably test FTP tonight to see if that's the case because running USB benchmarks on my rock64 show the speed maxes out.

The whole reason for me buying the rock64 was to try and get better copy speed across my LAN compared to my banana pi ultra m2.

Will try Bionic soon.

Nah... Samba is fine. You don't have to migrate to Bionic. Just try putting "noatime" option in your fstab and use EXT4 format instead. If you still don't get at least  70MB/s, check your Samba config file (/etc/samba/smb.conf).

[KernelPanic]

I have the same problem with mine. I did read that SAMBA can be the issue. Plenty of articles on the NET to try and assist, but still stuck around 40MB/s for my HDD.   I think people have said FTP is very quick, so will probably test FTP tonight to see if that's the case because running USB benchmarks on my rock64 show the speed maxes out.

The whole reason for me buying the rock64 was to try and get better copy speed across my LAN compared to my banana pi ultra m2.

Will try Bionic soon.

Nah... Samba is fine. You don't have to migrate to Bionic. Just try putting "noatime" option in your fstab and use EXT4 format instead. If you still don't get at least  70MB/s, check your Samba config file (/etc/samba/smb.conf).

Sadly reformatting to EXT4 and adding noatime didn't solve my issue, I was still getting poor performance. I've done some reading and noatime and it seem it would help on writes, but not much on reads (where I also have problems).

I did notice though that I was probably doing something probably pretty stupid... 

I was under the impression that it was possible to dualboot with SCard / eMMC. All these test I've been doing was with the latest ayufan bionic setup installed on the eMMC.
To test If I was having these problems with other images I installed them on the SDCard and put on the jumper, but at some point I noticed that my kernel wasn't changing along with the images. Also uninstalling the newest kernel and reinstalling the older kernels gave me the newest one with uname -a (even though in the /boot/ directory I had the older kernel files). As you can see I'm still a but of a noob with linux

So I've taken out the eMMC, reflashed the OMV version Luke suggested and things seem a lot better now.

Here's some testing I've done so far

Write:

Code:

root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"
root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# dd if=/dev/zero of=/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test/largefile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 8.77703 s, 122 MB/s

Read:

Code:

root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"
root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# dd if=/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test/largefile of=/dev/null bs=4k
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 6.32938 s, 170 MB/s

That's more like it!

Code:

root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
       Iozone: Performance Test of File I/O
               Version $Revision: 3.429 $
               Compiled for 64 bit mode.
               Build: linux

       Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                    Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                    Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                    Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                    Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                    Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                    Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                    Vangel Bojaxhi, Ben England, Vikentsi Lapa.

       Run began: Wed Apr 11 08:08:53 2018

       Include fsync in write timing
       O_DIRECT feature enabled
       Auto Mode
       File size set to 102400 kB
       Record Size 4 kB
       Record Size 16 kB
       Record Size 512 kB
       Record Size 1024 kB
       Record Size 16384 kB
       Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
       Output is in kBytes/sec
       Time Resolution = 0.000001 seconds.
       Processor cache size set to 1024 kBytes.
       Processor cache line size set to 32 bytes.
       File stride size set to 17 * record size.
                                                             random    random     bkwd    record    stride
             kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
         102400       4     6531     8253    17112    15665    11585     3837
         102400      16     5599    33859    55510    58457    23807    18672
         102400     512    84667   124221   122587   158883    79472    68095
         102400    1024    94451   111794   102468   127790    98707    69394
         102400   16384    87343   130649   131979   146383   149821    90956

Write seem a bit low, anything I'm doing wrong with the parameters im feeding iozone?

Tested with Samba a bit and downloading from Rock64 hovers around 80 - 100MB/s, uploading seems to be about the same.
A lot better than my earlier results!

In case it my help someone, this is how OMV mounted the drive:

Code:

LABEL=boot /boot/efi vfat defaults 0 0
# >>> [openmediavault]
/dev/disk/by-id/ata-ST4000DM004-2CV104_ZFN0B7VM-part1 /srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
# <<< [openmediavault]

and finally

Code:

root@rock64:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN0B7VM-part1/test# uname -a
Linux rock64 4.4.77-rockchip-ayufan-136 #1 SMP Thu Oct 12 09:14:48 UTC 2017 aarch64 GNU/Linux

Thanks everyone for helping, I'll probably stick with OMV since I really like the webUI