03-22-2016, 07:36 PM
(This post was last modified: 03-22-2016, 07:49 PM by falk.ben@gmail.com.)
(03-22-2016, 01:28 PM)Andrew2 Wrote:(03-22-2016, 11:20 AM)janjwerner Wrote: I respectfully disagree. The provided information with links to amazon is helpful.
Sure you will have variations between units of same type
Yeah, I could submit for the very same label (not card!) 'SanDisk Extreme Pro 8GB UHS-I' 3 different results that vary a lot. All tested in the same board. How's that possible? Since I sit in front of the 3 cards I could notice the differences (gold vs. black and obviously the two black ones from different years showing different random I/O performance).
Will users that submit results for their card take notice? Nope, since they don't even know that these differences exist when they don't own a couple of cards. So this will be a collection of performance numbers partially without meaning (or only of historical interest).
If I already bought a card I should test it always myself and never rely on results published in a spreadsheet somewhere since not performance numbers in a spreadsheet matter but my card's performance (see above: Which of the 3 results to choose from if users start to submit conflicting results? Most probably simply the first entry you stumble accross) .
It should be noted that you can buy fake cards everywhere and it's also not that hard to enter the words 'amazon fake micro sd card' or 'amazon counterfeit micro sd card' into a search engine to start to understand that the supply chain (where the fake products origin from) does not start at the retailer but somewhere in Asia. It's impossible for a big retailer to protect you from buying fake cards, the only chance you have is to immediately test the card you receive and return it asking for a refund when you got a fake card.
So this spreadsheet will be filled partially with meaningless results and as a user you can't rely on the results published there since they have zero relevance for the card you bought. So why looking into it anyway? If you search for a performant card (especially random I/O) then why searching through a spreadsheet collecting all sorts of questionable cards (or results) when there are also lists available that only collect top performers worth a buy? The most important factor for any SD/TF card used with an SBC is random I/O! Therefore it's close to moronic to choose anything else than the 5 top performers when it's about to buy a new card.
Regarding comparison of the results it's also pretty easy. Both RPi and Pine64 max out regarding sequential transfer speeds at approx. 22MB/s. If you see lower results then it's most probably the card's fault and if you see ~22MB/s then you know it's fast enough regarding sequential transfer speeds. Regarding random I/O it only depends on the card in question. In other words: The two tables containing well known top performers can be used for Pine64 without any doubt.
(03-22-2016, 11:45 AM)falk.ben@gmail.com Wrote: I think all these reasons taken together justify creating a new table of benchmarked cards. But, as I said, if you don't want to use it, no one is forcing you to.
I won't use this table for sure since I sometimes do benchmarking for a living. Without controlling every detail the chance that you collect numbers without meaning is pretty high, especially on Linux/Unix where background jobs might render benchmarks results useless easily.
People will submit results showing bad sequential/random I/O not caused by the card but since they played a bit around while testing, a cronjob in the background has been busy updating locatedb or the system is doing unattended package upgrades while they ran the single test execution they blindly trust in. Running benchmarks in a 'fire and forget' manner is always wrong but will be the default when random people submit random results.
I know that people love numbers (so easy to compare) and almost never think about the meaning of these numbers or whether they can be trusted or not. Or in other words: results that have been measured correctly (repeating the test at least 3 times and controlling setup/procedure using approriate tools -- iotop/iostat in this case) will be a rare exception.
I hope that sounds not rude but at least discouraging enough to stop the whole approach? :-)
So to summarize, it seems like your argument is that bad information is worse than no information.
I would simply argue that more information is better, and let everyone decide for themselves. I will add another column to the table with the username from this forum, so that people feel some sort of responsibility, and I will also be adding a short disclaimer to the top post here, so that everyone is aware of the pitfalls of collective benchmarking. I appreciate your criticism, and if you have benchmarking to add from those three cards with the same model name, I would greatly appreciate it being added to the table. I think it's good to know that the same model card could perform quite differently, depending on the batch.
(03-22-2016, 07:28 PM)janjwerner Wrote: I use:
Sony 32gb microSDHC Memory Card (SR32UY2A/TQ) works OK (16mb/s sequential read/write)
Sandisk Extreme 32GB U1 micro SD card (SAEMSD32GBQ) worked much better ( 23mb/s with sequential read / write)
Sony cards were $7 each few weeks ago, Sandisk was $25 two years ago.
cheers
Could you do 4K read/write test? For sequential read/write were you using hdparm or dd?