Product Idea: USB Flash Drives

[barray]

Quite frankly, I'd like the most a simple Git or Subversion repository hosted on our own, together with cgit or ViewVC, and accessed using everyone's favorite tools.  Optionally, there could be a read-only replica of the repository on GitHub.  By the time we get to writing actual code, I should be able to provide a hosted repository.

cgit looks cool, will have to check that out. I don't think it really matters all too much though and bare in mind, if our project suddenly gets referenced to by say Reddit and the server experiences a massive load, somewhere like GitLab would be able to handle it whereas our servers would be crushed. I understand the want to control the entire stack, but ultimately the decentralized nature of git means we'll have our own copies of the code in the worst case anyway.

Also, nobody hangs out in my self-served IRC channel especially for this project I guess only when we are official will Pine host a channel for us.

Yes, I saw your edit to the main wiki page and its later revert.  I agree that an "imaginary product" doesnt't have its place on the main wiki page in its current layout, but the PineFlash wiki page needs to be linked somewhere.  However, at the moment I'm really not sure how to tweak the layout of the main wiki page to achieve that.

Apparently nobody else really has any idea where to put it either. Their main concern if I understand correctly is that people believe it is some official Pine product ready to go into production. Normally the discussions we're having happen behind closed doors at pine when they are first designing out the hardware anyway. This would be an entirely new process for them.

Totally agreed.  Quite frankly, when it comes to automating the disconnect, it would be actually easier to build a robotic arm that would physically plug the device in and out of a USB port instead of a human.

Yeah this is what I also thought... USB is ultra sensitive to data line capacitance...

Umh, I'm not sure about that.  Even simple Micro SD cards are rather out of control when it comes to knowing that a write operation reported as successful has actually reached the underlying flash memory, which would make it possible to randomly cut the power or yank the card and lose no data.  Even rebooting the host can be problematic, because in many cases the reboot must include power cycling of the Micro SD card.  If the host uses the card at higher speeds and 1.8 V, the card must be power cycled to become initializable and usable by the boot loader after the reboot.

I think there are some things we can explore here. It may be difficult, but I believe we should be able to make some guarantees about the data, even if that means immediately ready it back.

Letting "smart" flash do its job for a certain period of time (with an additional supercap or anything else) and hoping that data actually reaches the flash still doesn't provide the required level of data reliability.  It would be a hit-and-miss approach, which would only provide a false sense of reliability.  To have reliable storage, in our case we need underlying flash that is dumb, in the sense of being raw and having no integrated logic.  In the case of raw flash, having a write operation that is reported back as successful actually means reaching the flash with the written data, which is what we need.

Not entirely true... Having it come back with 'success' just means that it "believes" it wrote something, but of course the memory cells are most vulnerable when their state is being changed and you could get electron leakage that flips the bit.

Having ZFS as an example, one of its requirements to provide data reliability is to have nothing between the filesystem and the actual drives.  In other words, no hardware RAID controllers (although there are actually no hardware RAID controllers) may be used, only raw drives.  Sure, drives also provide write caching on their own, which is a potential issue, but they can also reliably be told to flush the caches or to comply to a write barrier.  It took years to flesh out all kinds of bugs related to the management of write caches on HDDs.

ZFS was just an example of how software itself can be more robust. We'll have to think a lot more about exactly how to handle these edge cases and I really suspect hardware limitations will mostly force our hand. As long as the result is predictable, I think this is already pretty great. I've seen a device that attempt to write data on power down, the main-micro begins to crash before the flash does (this flash operated at lower voltages) and then it wrote semi-random data to a random address - not fun!

On the plus side, our job should be a little bit easier than that of spinning disks (fortunately). Read/write time should be constant and predictable, so it's just a case of a suitable algorithm - and much robust testing.

On a completely different note, I was thinking of some fun way to 'brand' the device. If we do stick with 'PineFlash', perhaps we could reference either Flash Gordon: https://www.imdb.com/title/tt0080745/ or The Flash: https://www.imdb.com/title/tt3107288/ ? I think both could be fun - the first is the savior of the Universe, the second is a super fast hero Either way, I'm feeling a red PCB with exposed copper could look epic! https://i.pinimg.com/originals/25/95/36/...8e2be8.jpg