zram is awesome! PBP is awesome! zram on PBP is awesome^2!

zram is awesome! PBP is awesome! zram on PBP is awesome^2! - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=111)
+--- Forum: Linux on Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=114)
+--- Thread: zram is awesome! PBP is awesome! zram on PBP is awesome^2! (/showthread.php?tid=9484)

zram is awesome! PBP is awesome! zram on PBP is awesome^2! - moonwalkers - 03-20-2020

This basically puts the amount of memory used at about 9.5GiB (used + DATA - TOTAL), that's on a system with only ~3.7GiB RAM available!

Code:
$ free -h; zramctl

 total        used        free      shared  buff/cache   available

Mem:          3.7Gi       3.5Gi        34Mi        84Mi       224Mi       114Mi

Swap:          14Gi       7.9Gi       7.0Gi

NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT

/dev/zram0 lzo-rle      14.8G  7.9G  1.9G  1.9G       6 [SWAP]

And the system is still surprisingly responsive for a low-power CPU.

RE: zram is awesome! PBP is awesome! zram on PBP is awesome^2! - Syonyk - 06-10-2020

Compressed swap is certainly awesome, but I would suggest looking very hard at zswap as opposed to zram. You get some of the same benefits, but a far more "swap-aware" compressed swap system than zram, and you avoid some of the problems with zram (at least if you have a traditional swapfile as well).

While zram is just a compressed block device, zswap is a compressed frontend to the disk based swap - it does require a swapfile on the disk, but it largely won't get used. When the kernel wants to evict a page from RAM to swap, it routes it through zswap, which checks for several things. If the page is a same-filled page (typically all 0s, but could be any repeating value), that gets stored and the page is freed without taking any swap space, because it's easy to recreate. The page is then run through a configurable compression algorithm (typically either lz4 or lzo). If it doesn't compress well enough, it just gets tossed out to the backing swap. After compression, the kernel will store either two (zbud) or three (z3fold) compressed pages in a single page of RAM. It's slightly wasteful in that a single page is rarely totally full of compressed data, but with the same filled page compression, I typically see 2.7x compression in zswap under normal running.

Where zswap is definitely superior, though, is that it has a LRU concept - least recently used. Old data can be pushed out of RAM to the backing file. If it's not been used in a long time, it's unlikely to be used again, so it can get evicted from RAM and free up that space. zram does have the writeback feature, but it's not page-aware - it can only write out whole pages of compressed data. And I'm not regularly seeing that feature enabled in the "swap on zram" configs. This, then, leads to a bit of a priority inversion. The oldest, longest-swapped out pages are likely to be stuck in zram, always taking memory, and reducing the amount of RAM available for active tasks. It's better than nothing, but under higher memory pressure, it ends up not working nearly as well.

So if you like zram, try zswap. It's built into most modern kernels, just enable it over in /sys/module/zswap/parameters/enabled, and tweak settings as you wish. You might try lzo as the compressor and z3fold as the zpool, if you're above about kernel 5.0 - there are bugs in z3fold before around 4.20 that cause weird lockups under high memory stress.

RE: zram is awesome! PBP is awesome! zram on PBP is awesome^2! - moonwalkers - 06-11-2020

Just to clarify - neither zram nor zswap is something new to me, I've used both for years. I'm also quite aware of non-mainline tech called zcache. The above post was merely expression of how impressed I was at how well something as low-powered as PBP works with zram, considering its benchmark scores are around the same level as Core 2 Duo, and it appears to be actually more responsive under the same conditions, which I guess is due to the fact that individual cores are perhaps less powerful, but there are three times as many, allowing to spread the load better.

As to zswap being better than zram - that is a claim I will contest. From my experience in both personal computing and working in big enterprise IT there are no one-fits-all solutions when you're trying to optimize performance, it all depends on usage patterns.

In my own experience, under comparatively low memory usage - oh yeah, zswap is great. And zram handles that situation just as well. Now, you don't get with zswap the situation where once you run out of RAM to keep all your compressed pages + stuff that cannot be paged out you're either dealing with priority inversion you've mentioned or your system slows down to a crawl while OOM killer tries to free up some memory. But in my experience if you're in that spot where zram still has enough compressed space to keep paging yet zswap already starts to fall back to on-disk swap, which I personally hit rather frequently on systems with <8GiB RAM with my usage patterns, zswap results in much worse responsiveness. I'm hypothesizing that the reason behind that worse performance is due to the fact that it has to decompress all those compressed pages before writing them to disk, causing even higher CPU load in addition to I/O contention with the apps I use. I've tried tweaking zswap in various ways, yet invariably I have been getting better performance with zram than with zswap for my usage patterns. But your mileage may vary.

RE: zram is awesome! PBP is awesome! zram on PBP is awesome^2! - Syonyk - 06-11-2020

I do wish zswap could write out the compressed pages - not sure how complex that would be to implement.

RE: zram is awesome! PBP is awesome! zram on PBP is awesome^2! - Surehand53 - 06-17-2020

(03-20-2020, 09:34 PM)moonwalkers Wrote: This basically puts the amount of memory used at about 9.5GiB (used + DATA - TOTAL), that's on a system with only ~3.7GiB RAM available!

Code:
$ free -h; zramctl total used free shared buff/cache available Mem: 3.7Gi 3.5Gi 34Mi 84Mi 224Mi 114Mi Swap: 14Gi 7.9Gi 7.0Gi NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT /dev/zram0 lzo-rle 14.8G 7.9G 1.9G 1.9G 6 [SWAP]

And the system is still surprisingly responsive for a low-power CPU.

Hi moonwalkers.

After your enthusiastic description I thought I try zram.
I have used this guide to set it up and it seems to work
https://www.techrapid.uk/2017/04/create-zram-memory-compression-on-arch-linux.html

Is that similar to what you are using ?
And more crucially, How do I get to the 9 Gi free mem. If I understand correctly I have 6 Gi from my guide ?

RE: zram is awesome! PBP is awesome! zram on PBP is awesome^2! - moonwalkers - 06-17-2020

(06-17-2020, 11:26 AM)Surehand53 Wrote:
(03-20-2020, 09:34 PM)moonwalkers Wrote: This basically puts the amount of memory used at about 9.5GiB (used + DATA - TOTAL), that's on a system with only ~3.7GiB RAM available!

Code:
$ free -h; zramctl total used free shared buff/cache available Mem: 3.7Gi 3.5Gi 34Mi 84Mi 224Mi 114Mi Swap: 14Gi 7.9Gi 7.0Gi NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT /dev/zram0 lzo-rle 14.8G 7.9G 1.9G 1.9G 6 [SWAP]

And the system is still surprisingly responsive for a low-power CPU.

Hi moonwalkers.

After your enthusiastic description I thought I try zram.
I have used this guide to set it up and it seems to work
https://www.techrapid.uk/2017/04/create-zram-memory-compression-on-arch-linux.html

Is that similar to what you are using ?
And more crucially, How do I get to the 9 Gi free mem. If I understand correctly I have 6 Gi from my guide ?

I used a bit different approach. My setup involves three files:

Code:
$ cat /etc/modules-load.d/zram.conf

zram

$ cat /etc/udev/rules.d/10-zram.rules

KERNEL=="zram0", SUBSYSTEM=="block", ACTION=="add", ATTR{initstate}=="0", PROGRAM="/bin/sh -c 'echo $(($(LANG=C free -k | sed -nr s/^Mem:\ *\([0-9]+\)\ +.*$/\\1/p)*4))KiB'", ATTR{disksize}="$result", RUN+="/sbin/mkswap $env{DEVNAME}",TAG+="systemd"

$ cat /etc/fstab

LABEL=pine-RootFS   /         ext4 defaults,relatime 0 1

PARTLABEL=pine-Boot /boot     ext4 defaults,relatime 0 2

PARTLABEL=pine-EFI  /boot/efi vfat defaults,relatime 0 2

/dev/zram0          none      swap sw                        0 0

This leaves compression method as default lzo-rle instead of lz4 - I prefer slightly better compression ratio of lzo over the performance of lz4, especially since I haven't noticed much perceived performance difference between the two. The tweaking of the zram swap size is done with the multiplier near the end of the PROGRAM expression, I currently use *4, which means four times the available RAM, but I've never really reached that usage level, maximum I was ever able to utilize is about 2.5xRAM, so 3xRAM will be more than enough. In other systems I've used 2xRAM, and typical setting in Chromebooks or Amazon Linux WorkSpaces, for example, is 1.5xRAM.

As to the "free memory" - just to clarify, you'll never get more free memory than what your PBP physically has, but zram can allow you to put more stuff in that memory by compressing that stuff. That means if the apps you run fill their memory pages with patterns that are not easily compressible by the lzo or lz4 algorithm you will run out of RAM even if you have "plenty free space" in your zram swap, your system will slow down and may even have OOM killer going on a hunt. But if you run apps that fill memory with patterns that can be easily compressed then you will achieve the result I've described above and maybe even better. The reason I use high zram to RAM ratio is because in my experience the system behavior is similar regardless of which way the system runs out of memory, swap space on zram first or RAM to allocate for zram first, but with bigger zram I get more leeway in case the data is well compressible (seems to be the case for browsing).