(01-19-2020, 05:15 PM)neilman Wrote: On my windows PC's I use a program called Macrium Reflect - it takes shadow copies of a hard drive, only saving "active sectors" and applies some compression before saving to an external hard drive.
With regards to block-level backups:
-
partclone is a tool that does block-level images, but tries to be a little bit smarter than plain simple `dd` and detects which blocks are actually used by data and only copies thoses and skips unused blocks (BTW: which means it cannot copy a corrupted file system, you need to fall back to partclone.dd or plain dd for that case). It supports multiple filesystem to detect used block, including all the common Linux ones. It can perform compression on the flight, and it can split it the backup in multiple files if you're writing it on something like FAT32 that doesn't support large files.
It's available on virtually all modern Linux distros, so you should definitely be able to install it on which ever distro you chose to boot on SD to backup your eMMC.
(It's also pre-installed on x86 desktops on the
System Rescue CD)
-
partimage roughly similar to partclone. A bit older and thus doesn't support as many filesystem (e.g.: BTRFS isn't supported).
Now back to DD and FSTRIM.
- If you use `fstrim` to zero out unusued flash blocks AND if the backup medium you are storing your backup is format in some unix-y filesystem - e.g.: EXT4, BTRFS, etc (and thus support sparse files), you can use `dd conv=sparse` parameter: it will leaves `holes` (make a sparse file) in the destination and thus the file will use only the backup media for the amount of data, not the total size of the eMMC.
NOTE: Not all uSD, USB, etc. support TRIM. Some uSD do, a lot of USB controller can't pass the command to the flash (some JMicron can with a firmware upgrade, USB3's UAS have a better chance of being able to pass the command). Nearly all SATA do. I'm ready to bet that the eMMC-to-USB adapter accessory of the Pinebook Pro can't and thus you need to do the TRIM from within the machine itself.
CAVEAT: Do not store a raw ReiserFS `dd` image on another ReiserFS partition, it makes runninf fsck impossible.
CAVEAT: `fstrim` actually doesn't physically write zeroes on blocks. It only tells the
flash controller "
this region of blocks isn't used anymore". The exact effect depends on implementation: flash chip is free to decide to immediately erase the block and return it to the free pool, or just take note that it doesn't need to keep that chunk of content whenever it performs wear leveling and/or read-modify-write cycles. What the block will return depends entirely on implementation too (some can even return the old data). In the specific case of eMMC, most of the implementation will either return a block with all bits set to 1 (what an erased NAND block actually looks like) or all bits set to 0 (how en empty block usually looks to the system). The `lsblk --discard` option can give you more insight about how discard is handled.
In the case of the eMMC that is pre-installed on PineBook Pro: TRIM-ed (or CMD38 in eMMC/SD parlance) erase blocks should show up as zero.
- Note that TRIM also works on thinly provisionned LVM partitions. And they will properly return zeroes for trimmed blocks.
- another better approach to zero fill, is to actually write zeroes.
zerofree is a linux-specific tool that is able to zero the empty blocks of EXT2/3/4 partitions.
Writing a large file filled with zero is stupid bruteforce way to do it (`cat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero`)
Note that these methods actually *write* data on the flash. Wether the flash controller is able to detect the zeros and react properly (performing its own TRIM internally) or whether it will actually physically write 'zeroes' and thus wear the whole flash depends on the flash chip. I have no idea what the Pinebook Pro's eMMC does.
- for compression I would suggest Zstd. It's available on nearly all modern distros, and it's way faster or way better than gzip.
needs to be simple clean easy to use