SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - Printable Version +- PINE64 (https://forum.pine64.org) +-- Forum: ROCKPRO64 (https://forum.pine64.org/forumdisplay.php?fid=98) +--- Forum: RockPro64 Projects, Ideas and Tutorials (https://forum.pine64.org/forumdisplay.php?fid=104) +--- Thread: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) (/showthread.php?tid=15944) Pages:
1
2
|
SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - GreyLinux - 02-02-2022 Hi All, So I've written another install tutorial this time using SkiffOS, the first one hopefully helped a few users. I discovered SkiffOS on the Rockpro64 software release page however I was surprised to see literally no posts in the forum about it. I contacted the developer on Discord and with his endless help and patience , I have a working server running Docker and several containers. UPDATE 19 JUNE 2022: After a lot of debugging and a lot of hard work on part @paralin1 , the Rockpro64 is on the 5.18 kernel and great news , anyone who uses the Marvell 88SE9230 SATA card its now working perfectly out of the box with SkiffOS, the udev rule is no longer required as its included within the SkiffOS Rockpro64 config UPDATE: @paralin1 is currently working on providing pre-built images , for users who want to try SkiffOS without compiling their own build, a link will be attached when the images are available SkiffOS Github page The Docker containers I have are
now this sounds like a lot of containers for an arm board, but at idle which is most of the time, the Rockpro64 sits at 1% CPU use. Now before I begin, SkiffOS describes itself as: Quote:SkiffOS is a lightweight operating system for any Linux-compatible computer, ranging from RPi, Odroid, NVIDIA Jetson, to Desktop PCs, Laptops (i.e. Apple MacBook), Phones (PinePhone), Containers, or Cloud VMs. It is: The main thing to focus on, I think is that it is based on Buildroot, this is an immutable OS, so you have to compile the build on your daily machine(i.e laptop or desktop) and add any additional software packages and configuration options at build time. The benefits of an immutable system are best described above, but the main thing is "once setup" you can reproduce the build with ease and it will barely need any tinkering once its up an and running, plus it provides the security that should anything go wrong a quick reboot will restore the System back to its initial boot state. SkiffOS has a "persist" partition that stores all the files and folders that you want to persist after a reboot. This can be where all the docker configuration files and states reside. SkiffOS also has the option for a CORE container this is a container that will provide an OS environment more familiar to most, making it easier to interact with the persist partition, you can choose between many core environments such as Alpine, Debian, Gentoo or even Ubuntu with a desktop environment. All this is best explained on the Github page, I however didn't need a core container as once setup the only interaction I need is with the containers themselves. So lets begin! to start with I would create a Projects Directory on you daily machine, preferably in you Home directory and then Git Clone the repo from Github into the projects directory. Code: mkdir Projects Now I created a configuration directory that has all my docker container configs, files and scripts, I called it my_docker_config this allows me to export it as an option when compiling SkiffOS during build time but of course you could create these files and folders and copy and paste their contents one at a time to the persist partition after first boot, but this defeats the benefit of having a reproducible build. So a little work now, to create a config directory will really pay off. I have a udev rule that allows my marvell 88SE9230 Sata card to work and a fan script that starts the fan and keeps it at a constant speed, these are best explained in my other tutorial, however these are the types of things I would also want to have in my configuration directory. I also have a couple of systemd .mount files to mount my ssd's on startup similar to how an "fstab" file would work. This is how my my_docker_config directory tree looks Code: my_docker_config you can see I keep my docker configuration file in mydocker a fan script under "opt" and the rest of my config in "etc" this is all whats called a root overlay , all these file and directories will be overlaid upon boot and be placed in their respective folders in the live environment . You can also see I have "authorized_keys" under skiff this is where you can place your public ssh keys , so you have ssh access upon boot. my udev rule named 99-marvell.rules looks like this (This udev rule is no longer required it is included in SkiffOS config for the Rockpro64, its shown to inform only): Code: ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x1b4b", ATTR{device}=="0x9230", RUN+="/bin/bash -c 'echo %k > /sys/bus/pci/drivers/ahci/bind'" My fan_script.sh looks like this: Code: #!/bin/bash this will of course need a "fan speed service" to start from boot and a sym link to the service in the multi-user.target.wants directory as shown in the directory tree the service aptly named fan-speed.service looks like this : Code: [Unit] one of my SSD mount files looks like this: (please note the name of the mount file must be the path to the mount point with each folder separated by a hyphen i.e /mnt/mydrive/here would be mnt-mydrive-here.mount) (also note that a sym link to the .mount file must also be in multi-user.target.wants directory) Code: [Unit] here is the basic config for Radicale : Code: [server] and the snikket.conf file please edit with your email and domain name for snikket (please leave the ports commented out until later) : Code: # The primary domain of your Snikket instance and the Caddyfile required for caddy Code: (log_common) { and finally my docker-compose.yml file for all my containers apart from Portainer which needs to access the docker daemon via a unix socket (please edit to suit your needs): Code: version: '3.7' after you have your configuration directory you can move it to /Projects/SkiffOS/configs/apps/ this will make it an exportable option in the next step. So on to the installation! Hopefully you still in the Projects directory with the freshly git cloned SkiffOS and your custom config directory moved to its place in the apps folder. As per the instruction on the SkiffOs github page you need to export all the options you want for the build, I have my board requirement, docker compose (which also installs docker) and my custom config directory. this looks like this : Code: cd SkiffOS OK so I should mention that if you require any additional software such as "htop" installed then now is the time to select it this is done by typing Code: make br/menuconfig please note! all these commands will only work if you are in the the SkiffOS directory located in the previously created Projects directory This make command will create an ncurses menu that will allow you to select additional software, system configs kernel options .etc The target packages option will allow you to select additional packages such as htop. The / key will allow you to search all the options and space will allow you to select the option current highlighted . (remember to save before you exit the ncurses menu) Now to finish up the build process. Code: make configure This last command will begin the compiling process, this can take some time depending on your hardware, so now is the time to get a coffee! (please note that when you recompile, for example after an update to SkiffOS this will be alot shorter timespan, as a previously compiled image will exist and only updated options and packages will be compiled ) To copy the compiled image to an SD card or emmc (although I currently haven't tested if emmc works) You will need to be root to perform these next commands Code: sudo bash (Additional note whilst using Arch I had no issue with the format command above, however after switching to a Ubuntu based OS I noticed it failed with an error, the solution was to edit he formatting script and add a P to every partition and the fatlabel ) Code: nano ~/Projects/SkiffOS/configs/pine64/common/scripts/format_sd.sh see the addition of the p after every curly bracket Code: echo "Formatting boot partition..." now your ready to boot, but before you do I would create the Duckdns or other DDNS provider entries for the services you will expose to the internet (Please see my other tutorial for assistance), in my case Home assistant , Radicale and the Snikket XMPP server, so 3 entries in total. I will point out for completeness sake that if your following along with this tutorial and you are wanting to trial Snikket as an XMPP server it is not recommended to use a Dynamic Domain Name Service, preferably you will require a static IP address from your internet provider and you should use the recommended DNS providers suggested on the Snikket website . I have a static IP but I am using Duckdns as a free DNS which works perfectly for my use case. When you boot for the first time it may take a few minutes to provide SSH, if it doesnt provide SSH after 15 minutes, the SSH keys provided in the config file are either incorrect or you have an issue and the SD has failed to boot. (Connecting a serial console could help you resolve your issue see here ) After you have successfully booted move your mydocker config directory to the persist partition Code: mv /opt/mydocker /mnt/persist/ now to get Portainer up and running Code: docker run -d -p 9443:9443 --name portainer \ You will need to setup Portainer to your requirements (This docker command will only get it installed and running) Next pull all the container images for your services (This command will only work if your in the directory where your docker-compose file is located) in my case: Code: cd /mnt/persist/mydocker/ So If your following along and are using the Radicale container, you will need to run a "one time" command to create a user password using htpasswd this can be done using a "temporary container", the container is removed after the command completes. Code: docker run -v /mnt/persist/mydocker/radicale:/radicale --rm -it alpine:edge sh -c "apk add apache2-utils && htpasswd -c /radicale/users calendar" next if your using the samba container for a NAS and your would prefer to use password hashes rather than a plain text password, then run this docker command for each user and fill in the requested details. This will print to screen a hash for the specified user which needs to be added to the docker-compose.yml file as shown above where it says ACCOUNT_username Code: docker run -ti --rm --entrypoint create-hash.sh servercontainers/samba Now is the time to open the required ports on your router, Caddy only requires 80 and 443 for the reverse proxy service however Snikket requires quite a few port as shown here Snikket requires a configuration file as shown above, however to use with a reverse proxy (Because Snikket gets its own lets encrypt certificates) you will need to change the ports in the .conf file (i.e uncomment) to something different from 80 and 443 which are in use by the Caddy container. Before you uncomment these ports in the config I would start the Snikket containers, with out the reverse proxy running, that way the Snikket server can obtain its own lets encrypt certs without issue, the certs will renew without issue, with the newly altered ports in my case 5080 and 5443, from now on, no need to edit again. Run these commands then edit the .conf file and uncomment the alternative ports. Code: docker-compose up snikket_proxy snikket_certs snikket_portal snikket_server OK so your almost there, If you have a backup of your Homeassistant install, now is the time to copy it to the homeassistant directory otherwise once started the home assistant container will be from scratch and will need to be setup from new. Finally ! Code: docker-compose up -d This will start the containers in daemon mode (i.e in the background) If all is working you should see the HomeAssistant login screen, the Radicale login screen and the Snikket admin login screen at the addresses you specified with you DNS. Of course you will see all other service via their local network address/IP I would now create an admin invite for yourself to access the Snikket admin page Code: docker exec snikket create-invite --admin --group default this will generate a URL for your first admin password setup All services will need configuring/setup to suit your needs but all containers should be running and healthy, this can be checked in the Portainer UI. Congrats!! as a final note! How to update SkiffOS, if there has been a new release or if you have changed the config on your build / require additional packages Code: cd Projects/SkiffOS Note! you may need to re-build fully if too much has changed (it which case the make clean command should be used before compiling): Code: make clean Then compile: Code: make compile Then push the updated image: Code: ./scripts/push_image.sh root@SKIFFOS_IP_ADDRESS RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - paralin1 - 02-03-2022 Thanks for this! Will work on adding a config layer to automate setting this up as well. RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - Rocklobster - 02-03-2022 Wonderful piece of work. Would you consider giving the guys over at Armbian a hand. I'm sure they could do with your help. RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - GreyLinux - 02-04-2022 (02-03-2022, 08:14 PM)Rocklobster Wrote: Wonderful piece of work. Would you consider giving the guys over at Armbian a hand. I'm sure they could do with your help. Hi Rocklobster, not sure if your replying to my install tutorial or @paralin1 excellent SkiffOS project? if it was to me, my previous tutorial uses Armbian, although it does need updating, as I no longer recommend SimpNAS for NAS software (its rarely updated) and I know there has been some changes in the Armbian infrastructure (not sure that the correct term). maybe I will spin up Armbian on the RockPro to update my previous tutorial but I don't think I would change from SkiffOS its perfect for my needs and works flawlessly. Thankyou for your review if it was aimed at me RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - Rocklobster - 02-04-2022 It was certainly aimed at you. Your credentials are in no doubt here and Armbian are crying out for people like you. Armbian has been the go to Distro for the RockPro64 board and I've no doubt someone like you would be a great asset to the project. RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - bavay - 03-15-2022 Just a quick question: I'm using the same udev rule for my Marvell 88SE9230 but then I get the following system message at boot: [ 17.721247] sysfs: cannot create duplicate filename '/devices/platform/f8000000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/remapped_nvme' [ 17.721273] CPU: 0 PID: 550 Comm: bash Not tainted 5.15.25-rockchip64 #22.02.1 [ 17.721283] Hardware name: Pine64 RockPro64 v2.1 (DT) If I remove the udev rule, I don't see my two disks connected to the Marvell 88SE9230 but I don't have this message, so this is really coming from there. Do you also see this kind of message? Do you have any idea what is the reason for it? Mathias PS: as a side note, when I do a "hdparm -t /dev/sdb1" I get around 250 MB/sec whereas when the same ssd was connected to a usb3 port on the RockPro64 (with a usb3<->sata adapter) it was more around 350 MB/sec. I find it a little surprising, I was expecting the same kind of real world speed (since 350 MB/sec would fit within sata3 bandwidth). RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - paralin1 - 03-16-2022 @Rocklobster with SkiffOS and Buildroot we are working towards the goal of an extremely minimal and, more importantly, reproducible system: flash & forget w/ some amount of pre-configuration baked in, and simplified over-the-air update. This is where it is different from armbian, which uses the usual approach of a package manager, mutable / filesystem, no pre-config, and full-disk images as the main backup / restore approach (checkpointing). RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - gabrielsr - 03-27-2022 Thanks, for the tutorial! I tried to complile with MacOS and within Ubuntu in VirtualBox without success. Installed elementary.os in a laptop and trying to compile there. Any chance of distributing a version already compiled for RockPro64? RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - GreyLinux - 04-01-2022 (03-15-2022, 03:26 AM)bavay Wrote: Just a quick question: I'm using the same udev rule for my Marvell 88SE9230 but then I get the following system message at boot: sorry for the delayed reply work lately has been insane, I'm not sure why this is happening have you tried the udev rule with only one drive or none connected , just the Marvell Pcie connected ? (03-27-2022, 08:02 AM)gabrielsr Wrote: Thanks, for the tutorial! I need to add a note to the Tutorial really, but @paralin1 is currently in the Process of creating pre-compiled Images, so users can trial SkiffOS RE: SkiffOS server with Docker ,NAS, Home assistant, Jellyfin, Snikket XMPP (tutorial) - bavay - 04-01-2022 (04-01-2022, 04:07 AM)GreyLinux Wrote: sorry for the delayed reply work lately has been insane, I'm not sure why this is happening have you tried the udev rule with only one drive or none connected , just the Marvell Pcie connected ? That's a good idea, I've not tried it yet, but I'll do it. So far, I've opened a bug report on the Marvell kernel driver bugzilla (see https://bugzilla.kernel.org/show_bug.cgi?id=215732). I'll comment more once I'll have tried with only one disk or even none connected to the adapter. |