microSD card performance comparison

[Andrew2]

I think the comparisons to odroid c2 are off topic.  Yes, emmc is faster.

You also link to SATA performance, but we don't have SATA on this board.

It's a bit sad that you don't WANT to understand the information provided. It was not about ODROID-C2 and eMMC it was about the simple fact that flash based media with more capacity uses internal parallelism and certain storage aspects will be several times faster when choosing a device with higher capacity. The link to the linux-sunxi wiki (where you would be able to find so much Pine64 information) was not about SATA but was about storage performance in general and important tweaks to improve performance.

Since you still want people contributing to a collection of absolutely meaningless numbers it's a bit useless to continue but I try it a last time. Please look at these iozone numbers that were measured in an controlled environment (using iostat to verify whether results are ok or have to be dropped):

Code:

                                                  random  random
    KB  reclen   write rewrite    read    reread    read   write
102400       4    2620    3385     7465     7486    7509    3499
102400       4    3600    3490     7520     7503    7532    3591

4k random read/write  7.5MB/s and 3.6MB/s! Compare with Jeff Geerlings's numbers please. None of the fast Samsung cards exceeded 3.0 MB/s for 4k writes and my card is neither a Pro+ nor an EVO+ but just a simple EVO, exactly the one he listed in the 6th row with 5.36MB/s and 1.05MB/s. Why does my EVO outperforms his by 350% when it's about random writes? Since I use the 64GB model and he just the 16GB. If you keep this in mind you start to understand his table a bit better and also that it's not about Pro+, EVO+ or EVO but that for these specific models and this specific use case (small random writes) the capacity of the card matters way more.

Why do I show two rows above and why does 4K sequetial writes differ that much? Since I took the minute and flashbench, then checked for erase block and page sizes (16MiB for my EVO and not 4MiB as with most SD cards) and created a second partition on the SD card aligned to the erase block size and also an ext4 FS on top with settings that guarantee optimal performance:

Code:

mkfs.ext4 -O^has_journal -E stride=2,stripe-width=2048 -b 4096 -L EVO_optimal /dev/sdb3
mount -t ext4 -O noatime,nodiratime,data=writeback /mnt/optimal /dev/sdb3

By creating an optimal aligned partition with optimal FS settings we get over 25% better performance scores in the benchmark above. But more importantly this will result in a way better performance in 'real world' scenarios when lots of stuff gets overwritten or deleted/added at the same time (and that applies especially to old/slow cards).

And there are a lot of additional tweaks that can't be seen in improved benchmark numbers but that make a huge difference in 'real world' situations, eg.

Code:

echo 8 >/proc/irq/$(awk -F":" "/sunxi-mmc/ {print \$1}" </proc/interrupts | head -n1 | sed 's/\ //g')/smp_affinity

which lets all SD card related IRQs be processed by the 4th CPU core instead of the 1st (where by default all IRQ handling happens and what will become a bottleneck when your Pine64 is really busy), by disabling journaling on the SD card (has its downsides too) and by heavily increasing the ext4 commit interval (Armbian uses 600 seconds here instead of the 5 sec ext4 default... and yes, this has also downsides)

IMO that's what benchmarking is for: Using always the same hardware running benchmarks to identify bottlenecks and performance factors and then both choose the appropriate hardware (in our case a larger SD card if we're interested in improved random I/O performance) and tweak settings instead. Or as someone else said it: Do active benchmarking instead of collecting numbers without meaning: http://www.brendangregg.com/activebenchmarking.html

[Tarjei85]

I think the comparisons to odroid c2 are off topic.  Yes, emmc is faster.

You also link to SATA performance, but we don't have SATA on this board.

It's a bit sad that you don't WANT to understand the information provided. It was not about ODROID-C2 and eMMC it was about the simple fact that flash based media with more capacity uses internal parallelism and certain storage aspects will be several times faster when choosing a device with higher capacity. The link to the linux-sunxi wiki (where you would be able to find so much Pine64 information) was not about SATA but was about storage performance in general and important tweaks to improve performance.

Since you still want people contributing to a collection of absolutely meaningless numbers it's a bit useless to continue but I try it a last time. Please look at these iozone numbers that were measured in an controlled environment (using iostat to verify whether results are ok or have to be dropped):

Code:

                                                  random  random
    KB  reclen   write rewrite    read    reread    read   write
102400       4    2620    3385     7465     7486    7509    3499
102400       4    3600    3490     7520     7503    7532    3591

4k random read/write  7.5MB/s and 3.6MB/s! Compare with Jeff Geerlings's numbers please. None of the fast Samsung cards exceeded 3.0 MB/s for 4k writes and my card is neither a Pro+ nor an EVO+ but just a simple EVO, exactly the one he listed in the 6th row with 5.36MB/s and 1.05MB/s. Why does my EVO outperforms his by 350% when it's about random writes? Since I use the 64GB model and he just the 16GB. If you keep this in mind you start to understand his table a bit better and also that it's not about Pro+, EVO+ or EVO but that for these specific models and this specific use case (small random writes) the capacity of the card matters way more.

Why do I show two rows above and why does 4K sequetial writes differ that much? Since I took the minute and flashbench, then checked for erase block and page sizes (16MiB for my EVO and not 4MiB as with most SD cards) and created a second partition on the SD card aligned to the erase block size and also an ext4 FS on top with settings that guarantee optimal performance:

Code:

mkfs.ext4 -O^has_journal -E stride=2,stripe-width=2048 -b 4096 -L EVO_optimal /dev/sdb3
mount -t ext4 -O noatime,nodiratime,data=writeback /mnt/optimal /dev/sdb3

By creating an optimal aligned partition with optimal FS settings we get over 25% better performance scores in the benchmark above. But more importantly this will result in a way better performance in 'real world' scenarios when lots of stuff gets overwritten or deleted/added at the same time (and that applies especially to old/slow cards).

And there are a lot of additional tweaks that can't be seen in improved benchmark numbers but that make a huge difference in 'real world' situations, eg.

Code:

echo 8 >/proc/irq/$(awk -F":" "/sunxi-mmc/ {print \$1}" </proc/interrupts | head -n1 | sed 's/\ //g')/smp_affinity

which lets all SD card related IRQs be processed by the 4th CPU core instead of the 1st (where by default all IRQ handling happens and what will become a bottleneck when your Pine64 is really busy), by disabling journaling on the SD card (has its downsides too) and by heavily increasing the ext4 commit interval (Armbian uses 600 seconds here instead of the 5 sec ext4 default... and yes, this has also downsides)

IMO that's what benchmarking is for: Using always the same hardware running benchmarks to identify bottlenecks and performance factors and then both choose the appropriate hardware (in our case a larger SD card if we're interested in improved random I/O performance) and tweak settings instead. Or as someone else said it: Do active benchmarking instead of collecting numbers without meaning: http://www.brendangregg.com/activebenchmarking.html

So what you are saying is that the proper formatting of the sdcard and as large as you can afford to go is the best sollution to get high random read wright speed?
Slunds interesting considering I got my self a SanDisk Ultra 128Gb sdxc1 card (SDSQUNC128GG6)

[Andrew2]

So what you are saying is that the proper formatting of the sdcard and as large as you can afford to go is the best sollution to get high random read wright speed?

Regarding the capacity of the card it depends on the card's controller. While it's known for some cards that their controller starts to do reads/writes in parallel when more flash chips are available cheap cards won't show this behaviour. Also there will be a maximum throughput so trying to double the speed by doubling capacity won't work infinite (see the aforementioned link to ODROIC-C2's eMMC behaviour to get the idea)

Here is some slightly outdated background information on the SD card market: http://www.bunniestudios.com/blog/?page_id=1022 (most obvious change back to then: you're able to buy SD/TF cards from Samsung now). Given these informations for me it makes absolutely no sense to buy anything else than either SanDisk (Extreme Pro or Extreme Plus) or Samsung (which label to choose depends on the use case). And if you care about longevity then the few bucks more for Extreme Plus or Pro+/EVO+ are a good invest.

Regarding proper settings there's a lot to do. Starts with identfying the internal behaviour of the SD card controller (page and erase block size) and then adjust partition and filesystem parameters to that. Then disabling ext4 journaling and switching to writeback helps a lot for specific use cases (but longsleep enabled the journal for a good reason: Since many Pine users simply have no clue what a filesystem is and that you should shutdown the Pine before powering off so that the filesystem is in a clean state after the next boot). And so you can tune other parameters as well (even exchanging the filesystem with for example btrfs with active transparent compression)

But the most important thing when it's about storage is: don't buy cheap crap. Buy recommended cards and test immediately whether you got fake/fraud storage media or not. And since Pine64's SD card implementation is unfortunately slower than necessary consider using USB connected fast storage also especially if you want to try 'Linux desktop' experiments.

[Tassadar]

So what you are saying is that the proper formatting of the sdcard and as large as you can afford to go is the best sollution to get high random read wright speed?

Regarding proper settings there's a lot to do. Starts with identfying the internal behaviour of the SD card controller (page and erase block size) and then adjust partition and filesystem parameters to that. Then disabling ext4 journaling and switching to writeback helps a lot for specific use cases (but longsleep enabled the journal for a good reason: Since many Pine users simply have no clue what a filesystem is and that you should shutdown the Pine before powering off so that the filesystem is in a clean state after the next boot). And so you can tune other parameters as well (even exchanging the filesystem with for example btrfs with active transparent compression)

How do you get these informations?

[simonritchie]

> I made yesterday a quick test with an old 64GB mSATA SSD lying around in this enclosure and moved everything except of the bootloader to it (easy with Armbian). Even without being able to benefit from UASP it's unbelievable how different desktop usage feels.

I agree. This thread shows some performance measurements that I did after connecting a solid state disk via the USB port. http://forum.pine64.org/showthread.php?t...7#pid12937. Reads were a little faster than to the microSD, but writes were about three times faster.