Product Idea: USB Flash Drives

[dsimic]

I've added SMART to the Wiki's suggested features, (with a probable pre-req of UASP).

SMART would be one of the must-have features, simply because we don't want to make another black box.  The internal statuses of the device should be made available to the host (and to the end-users) as much as possible, using already existing mechanisms such as SMART.

In regards to the copy on write, what I meant was that the flash is not used as simple addressable blocks. Since we have wear leveling, and spares, we have to have an internal "directory" which when I ask for from  block 357 it gives me logical block 357, and not the physical block 357 which might be for something else.

My thought is to make sure that this internal "directory" of logical blocks from the host, to physical blocks in flash, is kept consistent. Even if we have to re-write the "directory", (with version number so we know which is more current), into another place. So that the old "directory" is good, UNTIL the new "directory" is completely written.

This is just another confirmation of the need to use raw flash, instead of using "managed" flash.  Only raw flash can be told exactly what to do and when, instead of having the embedded flash microcontroller doing things nondeterministically on its own.  Using raw flash would make it possible to use the inherent CoW nature of flash to provide higher-level CoW, for the stored data.

In other words, combining the CoW nature of flash and our own wear levelling algorithms would make it possible to ensure guaranteed data reliability, and even to provide the ability for the host to define write barriers, which would trickle the data reliability up to the OS and filesystem level.  That would be awesome!

Last, many software & hardware development projects include test jigs. We may want to create a USB test jig that will remove power. Perhaps by relay to avoid the capacitance issue.  Or perhaps use an existing USB controlled relay with a special USB to USB adapter that cuts power.

As already described, cutting power to a USB device is doable on recent PC motherboards, so no additional hardware or a dedicated testbed would be required.  Though, that would cover just cutting the power to a USB device, instead of simulating the unplugging of a device as a whole.  However, having the ability to cut power on demand should be good enough for a while.

I was essentially told that anything put on the main page would be removed. I think we would need to ask before adding something, otherwise some wiki admin will just revert it again.

I've spent some more time tidying up the main wiki page, and this edit introduced a link to the PineFlash page.

I believe that between the filesystem and the flash control, there should be something that can be done in this space. But I guess this can also be part of the experimentation anyway.

Unfortunately, there's no "magic" there to rely upon, only the simple but fundamental rules to follow.  As an example, you can find quite a few descriptions of the real-word issues with using microSD cards in embedded systems, caused by their nondeterministic nature.

Eliminating the controller (a true "liar") from the above-described equation leaves the host to deal with low-level flash issues, but eliminates the possibility of having incomplete writes treated as complete.  Of course, the possibility for silent data corruption still remains, as a constraint of the flash memory technology, which can be mitigated using additional ECC data.

I'll have to look at this further. By the way, the chips linked in the wiki-page are automotive grade, so they have ECC in them too.

Using flash chips that provide the ECC functionality internally would be good, as long as it's still raw flash.

Inspired of course - not a direct clone. Also the PCB would be entirely hidden in the final casing for the device.

I see no possible copyright issues with the color of the PCB, and the association with Flash Gordon might be mentioned somewhere unofficially, as a funny note.   However, please keep in mind that PCB manufacturers charge much more for red PCBs than the green ones.