Anyone planning on ZFS or BTRFS for file systems?

[DrYak]

Has anyone here used ZFS or BTRFS on ARM64?

Been using Sailfish OS-based smartphone for quite a while, thus BTRFS a lot.
(Jolla1 phone uses BTRFS for it's root+home, exactly for the snapshotting reasons as it enables simpler "roll-back")
(BTRFS is still an option for microSD storage on Sailfish X on Sony Xperia X and XA2 and I use that).
I also use it a lot on Raspberry Pi (but it's not in-kernel, so root partition is still limited to F2FS for booting reasons) (also technically the RPi are still running 32-bit kernels, even on chipset where ARM64 is supported).

It basically works, specially on more modern kernels where the bugs have been ironed out.
Compared to ZFS it has the advantage to being mainline kernel supported.
Compared to ZFS it uses also a lot less RAM.

The caveats:

- we're speaking of embed devices with not a lot of eMMC space. Specially on older 16GB phone it was possible to run out of allocable chunks. It's a good advice to use "single" instead of "dup" for metadata. It's also good to regularily do some maintenance (balancing).
64GB and 128GB eMMC should be less prone to that. More modern kernels have better auto-balancing and auto-defrag capabilities.
Choose a distro that has spent the time to develop necessary maintenance tools. e.g.: openSuse and check their "btrfs-maintenance" package.
(You can set it to periodically "scrub", "balance" and TRIM the freespace).
(Though I've carved my own scripts for the Debian based Raspbian)

- it's a CoW system. You DO NOT run fsck on a cow system. You either simply roll back to an older still good copy (modern kernel semi automate that for you), or fall-back to "btrfs restore -sxmS"  to extract still readable data if the filesystem has become completely unmountable. Forget about fsck/btrfs repair. Not worth your time. (CoW and - in one case of hardware corruption - btrfs restore have saved me bigtime).

- it's a CoW system: on large files with random writes (like databases, like VMs, like torrents) the file can pretty quickly become a large maze of pointers to new copies. Recent kernels have auto-defrag as an option to Btrfs, though it's not as good (supposedly) as ZFS'. On the other hand, "chattr +C" is your friend when creating files to flag them as "CoW disabled/inplace modification" (Note: currently you can't flag an already existing file. You need to create a new one, flag it and then transfert the data).

- it's an embed system, you might get stuck with a slightly oldish kernel. Check what features are actually availble *and considered stable* in the kernel (According to tables on BTRFS' wiki) before turning them on. e.g.: Zstd might not be stable in the LTS kernel we'll get on PineBook Pro at the beginning.

- as of now RAID-5/6 still ARE NOT considered stable (According to tables on BTRFS' wiki) - though it's not easy to have enough devices to begin with on a laptop.

Don't be scared of the list, I'm just nit-picking on details.