Anyone planning on ZFS or BTRFS for file systems?

[Arwen]

These file systems are meant for scenarios where there is redundancy in the form of multiple drives.

Trying to apply ZFS or Btrfs to eMMC alone may be worse than not using either system - it could be all working fine one moment but corruption in the wrong place means total loss of everything and no way to fix it at all - in similar situations, other file systems could recover.

I can't speak about ZFS, but BTRFS is not only meant for redundancy. RAID1 and DUP are among the possibilities. But it's also about providing subvolumes, snapshots, CoW, compression, checksum on everything, etc.
All these valid features even on small devices (Jolla even used it for that exact puprose on their first smartphone).

Thankfully, due to the way BTRFS has data laid out there is no such thing as a single wrong place that loses everything. I've been through some flash media corruption (including hiqh quality SD from reputable brand names - but you now, bad luck happens), and each time I've still been able to recover nearly everything (btrfs restore is your friend).

Yes, ZFS has RAID-1, (aka Mirroring), DUP, (aka "copies=2"), subvolumes, (aka datasets), snapshots, CoW, compression, (can select different methods, by dataset), checksums on everything, (can select different algorythms, by dataset).

One thing that ZFS does, is have attributes stored with the subvolume, (aka dataset). That means you don't have to remember you choose LZ4 compression. It's always there, until you change it, ("zfs set compression=off rpool/home" for example).


There are ways that BTRFS can loose data. It's not a totally COW;

https://btrfs.wiki.kernel.org/index.php/..._rename.3F

For home use, I'd not worry about it. But, for data center use, (in reference to BTRFS in general, not BTRFS on Pinebook Pro), I'd consider this un-acceptable.

[Arwen]

...
There are ways that BTRFS can loose data. It's not a totally COW;

https://btrfs.wiki.kernel.org/index.php/..._rename.3F

For home use, I'd not worry about it. But, for data center use, (in reference to BTRFS in general, not BTRFS on Pinebook Pro), I'd consider this un-acceptable.

Oh, I forgot another way BTRFS and ZFS differ, BTRFS stores the checksum with the blocks, (data or metadata).

ZFS stores the checksum in the parent block. So that means ZFS stores data block checksums with the directory entries' metadata, and the directories' checksum with it's parent directory. And the root level directory in the Uber block, (top level file system descripter).

This difference is not too important until you run across mis-written data. (Probably so rare, most people never even heard of it.) With BTRFS, a scrub of the data may not detect this. But, ZFS would. Again, not to important for home use. And I'd return to using BTRFS on a Pinebook Pro without hesitation if ZFS was not usable on ARM64.

This became important to me when I selected the first 8TB disk of reasonable price, the Seagate Archive SMR, (Shingled Magnetic Recording). This type of disk ALWAYS relocates data blocks on write, (somewhat like flash does). Using ZFS was a no-brainer on that disk. It gave me both data validation, (for the more complex firmware Seagate had to use), and snapshots so I can store and release incremental backups easily.

[DrYak]

First, to mention that I've been running successfully Manjaro atop of a BTRFS root partition for quite some time now.

What's needed:

• edit /etc/mkinitcpio.conf to add btrfs into the initrd
  MODULES=(btrfs)BINARIES=("/usr/bin/btrfs")
• then run mkinitcpio to create a new initrd with the necessary drivers.
  
• backup your old EXT4 partition (if it is not on a different device, e.g.: SD card vs eMCC).
  
• create and format a new BTRFS partition
  
• (optionnally: create subvolumes to your liking)
  
• modify /boot/extlinux/extlinux.conf so that it points out to the new partition created before (e.g.: I called mine ROOT_MNJRO)
  root=LABEL=ROOT_MNJRO
• copy all the files from the old partition to the new one, preserving all permissions, ACL, xattr, hardlinks, etc.:
  rsync -avPSHAX --inplace /mnt/sdcard/ /mnt/emmc/

Done.

Oh, I forgot another way BTRFS and ZFS differ, BTRFS stores the checksum with the blocks, (data or metadata). {...}
This difference is not too important until you run across mis-written data. {...}  With BTRFS, a scrub of the data may not detect this.

Actually, BTRFS *will detect it, too*.
For BTRFS to not detect it, you would need to have the corrupted data somewhat magically matching the corrupted checksum on the corrupted disk block. It's not theoretically impossible but chances are *EXTREMELY unlikely* if you picked up a decent checksum algorithm when formatting your BTRFS drive.
You'll get error messages such as "checksum error, expected 'deadbeefh' got '00000000h' instead"

(source: my experience with a miswritten flash block on an SD card due to power-failure on my Raspberry Pi).

This became important to me when I selected the first 8TB disk of reasonable price, the Seagate Archive SMR, (Shingled Magnetic Recording). This type of disk ALWAYS relocates data blocks on write, (somewhat like flash does). Using ZFS was a no-brainer on that disk. It gave me both data validation, (for the more complex firmware Seagate had to use), and snapshots so I can store and release incremental backups easily.

Cue-in current backlash against WD for silently sneaking SMR drives in their NAS line-up which behave poorly on ZFS RAIDz2. :-D

(Of course, this is due to RAIDz2 and RAID6 need to do read/recompute parity/write cycles, whenever they write something. Which leads to write amplification by the underlying SMR relocation scheme. It's not inherent to ZFS, it's common on anything with parity (including mdadm RAID6) that doesn't have tiered caching like BCacheFS to group everything in a single append-write to the backed)

[Arwen]

@DrYak Thank you. When I tried BTRFS over the weekend, I missed the InitRD part. (When I used BTRFS years ago, I built it into the kernel, no need for an InitRD.)

So, adding that part to my prior instructions, works. Now I have the ability to perform upgrades in the same partition space. I use this on my kernel command line;

Code:

rootfstype=btrfs root=/dev/mmcblk2p8 rootflags=subvol=root/20200616

When I update, I will clone "root/20200616" and create a new sub-volume. Boot off it, and then perform the updates. After I have 4 or so OS sub-volumes, I'll delete the oldest. This gives me some backout plans, and some backups of OS files.

[DrYak]

@DrYak Thank you. When I tried BTRFS over the weekend, I missed the InitRD part. (When I used BTRFS years ago, I built it into the kernel, no need for an InitRD.)

Yup, the various make inird tools' magic autodetection of file systems is of no use... if there exists no BTRFS to be autodetected yet.
(We all make stupid mistakes: on my side I banged my head a bit, having completely forgotten to update the extlinux.conf)

(on the other hand switching the /boot partition from FAT32 to Ext4 (which is also compiled in by default in the current uboot) was a breeze)

I would have documented it earlier but... well... real-life and day job got in the way :-P

So, adding that part to my prior instructions, works. Now I have the ability to perform upgrades in the same partition space. I use this on my kernel command line;

Code:

rootfstype=btrfs root=/dev/mmcblk2p8 rootflags=subvol=root/20200616

When I update, I will clone "root/20200616" and create a new sub-volume. Boot off it, and then perform the updates. After I have 4 or so OS sub-volumes, I'll delete the oldest. This gives me some backout plans, and some backups of OS files.

You might be interested in playing with snapper, it is designed exactly for that and can partially automate your workflow.

I also have my own scrub/rebalance/trim script if you're interested.

[Arwen]

@DrYak, thanks for the pointer to snapper, but it will be easy enough to modify my old ZFS snap shot utility. And snapper is not designed for alternate boot environments. I also modified my scrubber utility.

Now I have to decide, (though it will probably take me weeks), whether to use the whole eMMC, (except for "/boot" and swap), as BTRFS. Then control the sub-volumes for things like "/home", (which I would use my snapshot script).

[DrYak]

Now I have to decide, (though it will probably take me weeks), whether to use the whole eMMC, (except for "/boot" and swap), as BTRFS. Then control the sub-volumes for things like "/home", (which I would use my snapshot script).

That's literally been my modus operandi on my pinebook pro and several other devices (e.g.: my workstation at work, etc.), and also been the default suggestion in openSUSE (and also on CentOS back when they still used BTRFS for snapshots, before abandoning it and moving to stratis).
Basically keep your root in one subvolume (the default one) and have everything else that you don't want to version (like home, opt, usr/local, srv, tmp, various entries in var, etc.) in a different subvolume that you won't snapshot.

A long time ago, I used to have a separate /home because:

• out of habit
  
• space management (overfilling /home won't prevent the system to run with an over filled /)
  (that was an important point before ENOSP got fixed).
  
• when BTRFS was young, out of fear of losing my /home if / got corrupted
  (Turns out: never got any major problem with BTRFS that couldn't be recovered by CoW -- or in the case of dead SD card, by btrfs restrore)
  

Nowadays, the only still valid reasons to use multiple partition is that currently you can't specify per-subvolume RAID levels (yet. it's on the devs todo list).
So if you want your root's data chunks at RAID0 for speed and your home's data chunks at RAID1 for resilience, multiple partitions is the only way to go.
(I do that on my working laptop and my homeworkstation).

But that requires multiple devices (or in case of "dup" a spinning rust HDD).

As you only have a single eMMC in your PBP, the whole point is moot.
(well, unless you manage to fit dual NVMe and a PXL chip within the form factor and the power envelope of the pbp ...)

And snapper is not designed for alternate boot environments.

On the other hand, there were some people on this forum toying with the idea of booting grub, and snapper does support grub. (It's been also be toyed by o-droid users, and even Suse made conferences about it)

@DrYak, thanks for the pointer to snapper, but it will be easy enough to modify my old ZFS snap shot utility.

Yup, we all like our home-made tools, hand carved with love. And with Bash/Perl/whatever.