12-17-2018, 06:44 AM
I bought a RockPro64 and am trying to build a budget 10GbE storage.
I plugged an AQC107 based ASUS 10GbE card (XG-C100C) into its PCIEx4 adapter and Samsung 860EVO 1TB via USBC-SATA conversion cable (lsusb shows "ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge"), and no problem mounting it.
The image I chose this time is stretch-minimal-rockpro64-0.7.11-1075-arm64 and the driver for the card is Atlantic driver (Atlantic-2.0.15.0).
I downloaded the driver from Aquantia's website and had no problem building and installing.
After 'modprobe atlantic' the board immediately recognized the card as enp1s0. Then its idle power consumption (with all devices active) is around 10W, whose almost half the 10GbE card accounts for.
After changing its MTU from 1500 (default) to 9000, I performed iperf and I got a result around 2.70Gbps, though it seems that the result is not stable, sometimes the value going down to 2.30Gbps or lower.
I don't know why but manually assining iperf processes to a specific core (taskset -c 5 iperf -s, for example) seems to offer better and stable results that is 3.00Gbps.
At this point the effective speed as a NAS (sharing the SSD through SMB and mount it from a Windows client, transferring large data manually and observe how much it takes to complete) is approx. 330MB/s for read and write, and 4k random read/write is 7MB/s and 4KB Q8T8 and 4K Q32T1 for read are both 117MB/s, while it's both 100MB/s for write, says CrystalDiskMark 6.0.2. The peak power consumption is just 15.0W, so it's very power-efficient compared to x86-based 10GbE systems.
This is a great gain to have, considering the limit ROCK64 has.
Currently I have an impression that RockPro64 is an ideal solution for HDD-based NAS, and it would be fun if I would buy more boards and build a distributed system. But it's a little slow for SSD-based NAS, especially when looking at random read/write values.
I wonder if there is anyone who have had more comfortable results with 10GbE NICs and SSDs, to know whether I can get more out of this board or I've reached the limit.
Launching iperf multithreaded (-P6) greatly increased total bandwidth from 3Gbps at most to over 9Gbps, but after all SMB is not multithreaded and what matters most is achieving higher single-thread performance for personal use.
I admit there is room for optimization, so I will post if I have a significant improvement in performance.
I plugged an AQC107 based ASUS 10GbE card (XG-C100C) into its PCIEx4 adapter and Samsung 860EVO 1TB via USBC-SATA conversion cable (lsusb shows "ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge"), and no problem mounting it.
The image I chose this time is stretch-minimal-rockpro64-0.7.11-1075-arm64 and the driver for the card is Atlantic driver (Atlantic-2.0.15.0).
I downloaded the driver from Aquantia's website and had no problem building and installing.
After 'modprobe atlantic' the board immediately recognized the card as enp1s0. Then its idle power consumption (with all devices active) is around 10W, whose almost half the 10GbE card accounts for.
After changing its MTU from 1500 (default) to 9000, I performed iperf and I got a result around 2.70Gbps, though it seems that the result is not stable, sometimes the value going down to 2.30Gbps or lower.
I don't know why but manually assining iperf processes to a specific core (taskset -c 5 iperf -s, for example) seems to offer better and stable results that is 3.00Gbps.
At this point the effective speed as a NAS (sharing the SSD through SMB and mount it from a Windows client, transferring large data manually and observe how much it takes to complete) is approx. 330MB/s for read and write, and 4k random read/write is 7MB/s and 4KB Q8T8 and 4K Q32T1 for read are both 117MB/s, while it's both 100MB/s for write, says CrystalDiskMark 6.0.2. The peak power consumption is just 15.0W, so it's very power-efficient compared to x86-based 10GbE systems.
This is a great gain to have, considering the limit ROCK64 has.
Currently I have an impression that RockPro64 is an ideal solution for HDD-based NAS, and it would be fun if I would buy more boards and build a distributed system. But it's a little slow for SSD-based NAS, especially when looking at random read/write values.
I wonder if there is anyone who have had more comfortable results with 10GbE NICs and SSDs, to know whether I can get more out of this board or I've reached the limit.
Launching iperf multithreaded (-P6) greatly increased total bandwidth from 3Gbps at most to over 9Gbps, but after all SMB is not multithreaded and what matters most is achieving higher single-thread performance for personal use.
I admit there is room for optimization, so I will post if I have a significant improvement in performance.
