Product Idea: USB Flash Drives

[dsimic]

cgit looks cool, will have to check that out. I don't think it really matters all too much though and bare in mind, if our project suddenly gets referenced to by say Reddit and the server experiences a massive load, somewhere like GitLab would be able to handle it whereas our servers would be crushed. I understand the want to control the entire stack, but ultimately the decentralized nature of git means we'll have our own copies of the code in the worst case anyway.

Too high load on the server would actually be a good problem to have.  I wouldn't worry too much about it for now, though.

Apparently nobody else really has any idea where to put it either. Their main concern if I understand correctly is that people believe it is some official Pine product ready to go into production. Normally the discussions we're having happen behind closed doors at pine when they are first designing out the hardware anyway. This would be an entirely new process for them.

That's an accurate description of the issue at hand.  Maybe we could put the link into the "Community and Support" section on the main wiki page, as an example of the community collaboration and effort?  That might even attract the attention of Pine64.

Umh, I'm not sure about that.  Even simple Micro SD cards are rather out of control when it comes to knowing that a write operation reported as successful has actually reached the underlying flash memory, which would make it possible to randomly cut the power or yank the card and lose no data.  Even rebooting the host can be problematic, because in many cases the reboot must include power cycling of the Micro SD card.  If the host uses the card at higher speeds and 1.8 V, the card must be power cycled to become initializable and usable by the boot loader after the reboot.

I think there are some things we can explore here. It may be difficult, but I believe we should be able to make some guarantees about the data, even if that means immediately ready it back.

Meh, not even doing a repeated read from "smart" flash would guarantee anything, because the embedded flash controller could simply return the buffered data that is still in transit to the underlying flash memory. 

Some industrial full-size SD cards mitigate the issue by including supercaps that provide the emergency power required to actually complete all write operations that were acknowledged as successful, which is the same approach as in enterprise SSDs.  However, there's simply no space for such supercaps in Micro SD cards or SPI flash chips, which makes them inherently unreliable.

Letting "smart" flash do its job for a certain period of time (with an additional supercap or anything else) and hoping that data actually reaches the flash still doesn't provide the required level of data reliability.  It would be a hit-and-miss approach, which would only provide a false sense of reliability.  To have reliable storage, in our case we need underlying flash that is dumb, in the sense of being raw and having no integrated logic.  In the case of raw flash, having a write operation that is reported back as successful actually means reaching the flash with the written data, which is what we need.

Not entirely true... Having it come back with 'success' just means that it "believes" it wrote something, but of course the memory cells are most vulnerable when their state is being changed and you could get electron leakage that flips the bit.

The leakage you mentioned actually belongs to a whole other category of issues, usually called bit rot or silent data corruption.  It is actually unrelated to the acknowledged writes that end up not reaching the underlying flash memory.  I'll try to explain this further.

There are three "players" in this game: the flash memory, the embedded flash controller, and the host.  When the flash "lies" to the controller, by experiencing the effects of cosmic rays or a defective cell, it is the silent data corruption.  In this case, the flash actually doesn't "lie", it's instead simply unaware of the changes to the data.  When the controller "lies" to the host, as a result of performing the wear leveling or as just an attempt to improve performance, it is the write that hasn't actually reached the flash memory.  In this case, the controller really "lies", by being fully aware of the data not reaching the underlying flash.

Eliminating the controller (a true "liar") from the above-described equation leaves the host to deal with low-level flash issues, but eliminates the possibility of having incomplete writes treated as complete.  Of course, the possibility for silent data corruption still remains, as a constraint of the flash memory technology, which can be mitigated using additional ECC data.

On a completely different note, I was thinking of some fun way to 'brand' the device. If we do stick with 'PineFlash', perhaps we could reference either Flash Gordon: https://www.imdb.com/title/tt0080745/ or The Flash: https://www.imdb.com/title/tt3107288/ ? I think both could be fun - the first is the savior of the Universe, the second is a super fast hero Either way, I'm feeling a red PCB with exposed copper could look epic! https://i.pinimg.com/originals/25/95/36/...8e2be8.jpg

It would be fun, but receiving a copyright infrigement notice would be much less fun.