Sopine Fedora 32 on clusterboard w/networking

[revoman]

Just jotting down some notes, and a question after going through a painful process of trying to get a non-debian derivative Linux OS on my cluster.   

After some twists and turns I finally came back around to a post I saw regarding the arm-image-installer and a F30 image.  

I run Fedora on my workstation, so  all of the instructions assume a fedora build box.

1. sudo dnf install arm-image-installer
   
2. downloaded the raw image file from the Fedora download site and selected Fedora 32: Raw image for aarch64
   
3. Write the F32 aarch64 image to an SD card using the following command
   sudo arm-image-installer --addconsole --addkey ~/.ssh/id_rsa.pub --relabel --resizefs --image=Fedora-Minimal-32-1.6.aarch64.raw.xz --media=/dev/sdc --target=sopine_baseboard
   
4. Move card to sopine board
   
5. Insert sopine into clusterboard
   
6. Connect serial console to pins 6,7,8
   extracted from https://forum.pine64.org/attachment.php?aid=1111 and somewhere else
   
7. Connect to serial console via
   screen /dev/ttyUSB0 115200
   
8. Apply power to clusterboard - and prepare to press a key to "stop autoboot"
   
9. Press the space bar until
   Hit any key to stop autoboot:  0
   appears and you are presented with a prompt `->`
10. At this prompt execute this command and hit enter 
    mi w 1c30000 1f 7; mi w 1c30000 1e a4 ; mi w 1c30000 1c b573 ; mi w 1c30000 1f 0
    found at https://forum.pine64.org/showthread.php?tid=6036
11. Reset the board, either by typing reset or power cycling or the reset button.
    This often does not work the first time.
    
12. Wait for the kernel to load and Fedora 32 to finish initializing
    
13. Answer the remaining installation questions
    
14. Finish and log in.  
    
15. nmcli` should show an active eth0 device
    
16. ip addr should show an IP address from your DHCP server
    
17. Never remove power or reset the clusterboard. 
    

So.  After all that, I still get boot errors where the boards try to boot from the mmc before it has finished initializing causing the kernel or vmlinuz images to be unavailable.
Also, if I power cycle the board, the setting from #10 gets erased.

Question to the community.  How do I make `mi w 1c30000 1f 7; mi w 1c30000 1e a4 ; mi w 1c30000 1c b573 ; mi w 1c30000 1f 0` persistent?  Is this u-boot?  is this EFI?  I am a bit out of my league here.

Thanks in advance,
-Nate R

---

...
Question to the community.  How do I make `mi w 1c30000 1f 7; mi w 1c30000 1e a4 ; mi w 1c30000 1c b573 ; mi w 1c30000 1f 0` persistent?  Is this u-boot?  is this EFI?  I am a bit out of my league here.

Thanks in advance,
-Nate R

Found the answer to my own question.  Quite a crazy journey, but now I have fully working Fedora 32 sopine compute modules!  Now to k8s it.

The key came in this link: https://github.com/linux-sunxi/u-boot-sunxi/wiki
Turns out the u-boot process looks for a `boot.scr` file on the first fat partition for running scripts.  There was some pretty crazy dynamic variables going on, so it took a minute to get right.   Here's what I ended up with.

First fat boot device is mounted under /boot/efi
cd /boot/efi
Create the text file /boot/efi/boot.txt
Fill it with the u-boot commands you wish to execute. (note, current scripting continues the boot after these, so no need to re-run bootcmd)

Code:

echo starting eth0 fix
mi w 1c30000 1f 7
mi w 1c30000 1e a4
mi w 1c30000 1c b573
mi w 1c30000 1f 0
echo fixed eth0

Now comes the magic.  Use the command mkimage to turn it into a binary file u-boot can handle.

Code:

mkimage -A arm -T script -O linux -d boot.txt boot.scr

Reboot and profit.

-Nate Revo

[maya.b]

Hiya,

All of this has become easy and just works with F33 (or at least hte IoT version) for aarch64. I currently have 28 nodes on 4 clusterboards up and running without issue (apart from ones I create for myself)

M.

[revoman]

Hiya,

All of this has become easy and just works with F33 (or at least hte IoT version) for aarch64. I currently have 28 nodes on 4 clusterboards up and running without issue (apart from ones I create for myself)

M.

Thanks for the update.   Do you have any PCI cards in your boards?  The other issue I had was Fedora intentionally disabled the PCI-e slot.

-Nate

[langerma]

Hiya,

All of this has become easy and just works with F33 (or at least hte IoT version) for aarch64. I currently have 28 nodes on 4 clusterboards up and running without issue (apart from ones I create for myself)

M.

can you just take the raw image and write it to a sd card without this whole arm-installer thingy?

br.
Markus

[poVoq]

Seems to work with Fedora 37 IoT by just using the arm-installer to write the raw image on a SD card. Don't forget to add the SSH pub key as the IoT raw images do not allow root login with password. My Sopine modules in the Clusterboard boot up fine and get a IP address from DHCP.

I used this command to write the raw image on my SD cards:

Code:

sudo arm-image-installer --resizefs --image=Fedora-IoT-37-20221118.0.aarch64.raw.xz --media=/dev/sdc --target=sopine_baseboard --addkey ~/.ssh/id_rsa.pub

The Fedora IoT 38 Beta raw image seems to be broken though, at least my first attempt with it didn't work.

[langerma]

Seems to work with Fedora 37 IoT by just using the arm-installer to write the raw image on a SD card. Don't forget to add the SSH pub key as the IoT raw images do not allow root login with password. My Sopine modules in the Clusterboard boot up fine and get a IP address from DHCP.

I used this command to write the raw image on my SD cards:

Code:

sudo arm-image-installer --resizefs --image=Fedora-IoT-37-20221118.0.aarch64.raw.xz --media=/dev/sdc --target=sopine_baseboard --addkey ~/.ssh/id_rsa.pub

The Fedora IoT 38 Beta raw image seems to be broken though, at least my first attempt with it didn't work.

yes i tried it with server 37 and 38 without resizefs flag it is working fine :-) so all good. i just want to have the server version running as i am going to utilize the nodes with nomad and docker to use csi volumes. with podman this is not working though. but is there a way to skip the srial installation method and fully configure the image with a textfile or sth. like this?

br. markus

[poVoq]

The non IoT aarch64 raw images should work as well.

Once you installed it on one SD card with the Fedora installer you can use dd to make a disk image yourself and write that to other SD cards, but I don't really see the benefit.

[langerma]

well i do not want to go throught the installation routine on 45 sdcards :-) thats all.