Backup the EMMC
(01-19-2020, 05:15 PM)neilman Wrote: On my windows PC's I use a program called Macrium Reflect - it takes shadow copies of a hard drive, only saving "active sectors" and applies some compression before saving to an external hard drive. The file it saves is later accessible for individual file retrieval or can rebuild a complete new drive from the saved image.

Is there anything vaguely like that in "ARM" land?

Doesn't my posting come pretty close to that?
  • writing and deleting zeros onto the filesystem makes it highly compressable ... just like "saving active sectors"
  • you can loopback mount the dd'ed image (it might require some tricks to work around the gziped part and so you may want to temporarily gunzip it before loopback mounting) and extract files.
If you have util-linux,, there is fstrim (root) which will work on eMMC
(will NOT work on uSD or usb sticks)
Does a fine job zeroing, don't overuse
If you were trying to backup a running system, unless sudo there
would be all kinds of access denied messages, wouldn't work so well
And, for that matter, why wouldn't you back up to a usb stick,
which you could easily put into any computer, no need to unzip
to get a file or many off, just copy onto stick beside backup
As for size, well the gz will be a lot smaller, only if the expanded contents are larger
than the target would there be a problem. Yes a straight dd without a zip in the middle
stands a risk of being too large,, you could use gparted to shrink the donor first, 1-200M
There could be a small problem with a gpt type disk, gdisk can repair the backup table
(01-19-2020, 05:15 PM)neilman Wrote: On my windows PC's I use a program called Macrium Reflect - it takes shadow copies of a hard drive, only saving "active sectors" and applies some compression before saving to an external hard drive.

With regards to block-level backups:

- partclone is a tool that does block-level images, but tries to be a little bit smarter than plain simple `dd` and detects which blocks are actually used by data and only copies thoses and skips unused blocks (BTW: which means it cannot copy a corrupted file system, you need to fall back to partclone.dd or plain dd for that case). It supports multiple filesystem to detect used block, including all the common Linux ones. It can perform compression on the flight, and it can split it the backup in multiple files if you're writing it on something like FAT32 that doesn't support large files.
It's available on virtually all modern Linux distros, so you should definitely be able to install it on which ever distro you chose to boot on SD to backup your eMMC.
(It's also pre-installed on x86 desktops on the System Rescue CD)

- partimage roughly similar to partclone. A bit older and thus doesn't support as many filesystem (e.g.: BTRFS isn't supported).

Now back to DD and FSTRIM.

- If you use `fstrim` to zero out unusued flash blocks AND if the backup medium you are storing your backup is format in some unix-y filesystem - e.g.: EXT4, BTRFS, etc (and thus support sparse files), you can use `dd conv=sparse` parameter: it will leaves `holes` (make a sparse file) in the destination and thus the file will use only the backup media for the amount of data, not the total size of the eMMC.
NOTE: Not all uSD, USB, etc. support TRIM. Some uSD do, a lot of USB controller can't pass the command to the flash (some JMicron can with a firmware upgrade, USB3's UAS have a better chance of being able to pass the command). Nearly all SATA do. I'm ready to bet that the eMMC-to-USB adapter accessory of the Pinebook Pro can't and thus you need to do the TRIM from within the machine itself.
CAVEAT: Do not store a raw ReiserFS `dd` image on another ReiserFS partition, it makes runninf fsck impossible.
CAVEAT: `fstrim` actually doesn't physically write zeroes on blocks. It only tells the flash controller "this region of blocks isn't used anymore". The exact effect depends on implementation: flash chip is free to decide to immediately erase the block and return it to the free pool, or just take note that it doesn't need to keep that chunk of content whenever it performs wear leveling and/or read-modify-write cycles. What the block will return depends entirely on implementation too (some can even return the old data). In the specific case of eMMC, most of the implementation will either return a block with all bits set to 1 (what an erased NAND block actually looks like) or all bits set to 0 (how en empty block usually looks to the system). The `lsblk --discard` option can give you more insight about how discard is handled.
In the case of the eMMC that is pre-installed on PineBook Pro: TRIM-ed (or CMD38 in eMMC/SD parlance) erase blocks should show up as zero.

- Note that TRIM also works on thinly provisionned LVM partitions. And they will properly return zeroes for trimmed blocks.

- another better approach to zero fill, is to actually write zeroes.
zerofree is a linux-specific tool that is able to zero the empty blocks of EXT2/3/4 partitions.
Writing a large file filled with zero is stupid bruteforce way to do it (`cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero`)
Note that these methods actually *write* data on the flash. Wether the flash controller is able to detect the zeros and react properly (performing its own TRIM internally) or whether it will actually physically write 'zeroes' and thus wear the whole flash depends on the flash chip. I have no idea what the Pinebook Pro's eMMC does.

- for compression I would suggest Zstd. It's available on nearly all modern distros, and it's way faster or way better than gzip.
Sometimes fstrim takes 30 seconds to complete, I think it must be doing some writing in that time
But you are right, that is not long enough to zero all of the drive, is it writing its internal data structures?
I am not aware of any uSD that can be 'trimmed' , typical response is "command unsupported"
The "SD Card formater" from the SD foundation, running in win can do this, is there a linux tool?
And I'm wrong, time has moved on, badly needed and I am glad to be wrong here
In both cases, uSD and eMMC fstrim takes up to a minute, green led blinks, think it is writing
we need just a simple app like raspberry have that's the end the point my friends Wink needs to be simple clean easy to use
Nearly everyone will tell you it is a bad idea to back up a running system, maybe you know better
Raspberry had NO choice, only one boot medium, so, better than nothing
I'm not sure how 2 simple lines are hard......and only one at a time
To be sure, you have to get them right, or it will not be good
Note that taking a backup of eMMC with dd is the fast operation. Writing it back is a lot slower than reading it. Taking a backup of the home dir and then reinstalling the OS from scratch is faster.

Possibly Related Threads…
Thread Author Replies Views Last Post
  U-Boot with direct NVMe boot support for eMMC/SPI Flash pcm720 118 94,553 06-08-2021, 11:04 PM
Last Post: calinb
  UK Pinebook Pro for sale (UK buyers only, 64GB, eMMC reader, NVME add-on) Z0rua 0 136 06-01-2021, 12:51 PM
Last Post: Z0rua
  I am ready to accept that there may be a problem with my emmc. djhnsn 4 392 05-18-2021, 12:25 AM
Last Post: djhnsn
  Wiped EMMC, forgot do write new image, won't recognize EMMC zeno 4 471 05-11-2021, 01:53 PM
Last Post: wdt
  Need Advice for Wiping eMMC on Dead LCD Pinebook Pro calinb 9 743 05-10-2021, 12:49 PM
Last Post: calinb
  eMMC not gets detected by initramfs? overlisted 6 1,541 05-09-2021, 09:08 PM
Last Post: djhnsn
  Happy eMMC and SDcard OS usage for Pinebook Pro Newbies... Paulie420 40 19,166 02-27-2021, 11:01 AM
Last Post: wdt
  Pinebook Pro 128GB EMMC for sale **sold** dgkPBP 7 1,719 01-15-2021, 08:48 AM
Last Post: dgkPBP
  Booting to sdcard stopped working after emmc flash techiedog 3 1,109 12-31-2020, 06:30 PM
Last Post: Anarethos
  Flash non-bootable eMMC? midnightcheese 2 646 12-24-2020, 08:01 AM
Last Post: midnightcheese

Forum Jump:

Users browsing this thread: 1 Guest(s)