(06-25-2022, 12:21 PM)adamfowleruk Wrote: Hello all,

I've recently taken delivery of a DeskPi Super6C motherboard that can house 6x RPi CM4 on one side, and 6x NVMe SSD on the other side: https://deskpi.com/collections/deskpi-super6c/products/deskpi-super6c-raspberry-pi-cm4-cluster-mini-itx-board-6-rpi-cm4-supported

The CM4s are still hard to come by, and so I was thinking about trying to get one of the SOQuartz integrated with this mini-ITX sized host board. Before I buy a bunch of SOQuartz though I wanted to ask if the community thought this a crazy idea or not?

I'm very happy hacking around with Manjaro (I use it as a daily driver on my personal machine), and using serial access to machines, playing around with dts files, kernels, and so on for embedded, so don't think there's a huge issue there for me. I note that a few people are collaborating on SOQuartz on RPi CM4 host boards already: https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/336

Does anyone have any strong feelings on my above pairing at all? Is it a worthwhile project?

(Background: I'm looking to build a relatively low power multi-node cluster for lightweight Ubuntu (and Ceph) eventually with BYOH Kubernetes on top.)

Thanks in advance for all opinions and advice.

• no UART, which is very unfortunate since you'll most likely need this in some capacity
  
• SD card on the bottom, which means its harder to access, which you have to do more on the SOQuartz unless you want to bother with Rockchip's USB download mode that I don't think we've documented on the wiki anywhere
  

• SATA for each board (removes the USB 3.0 capability from the monitor board)
  • Allows to use this as a distributed (node redundancy rather than block device redundancy) storage cluster (Ceph, GlusterFS) with an HDD cached by an SSD. Imagine 6 SOQuartzes, each with a 18 TB drive (like $315 each right now) and a 500GB TLC NVMe SSD (like $50 now) for caching with bcache. At 2x redundancy, that leaves you with 54 TB of SSD-cached storage for like $2190 for the storage and around $600 for SOQuartz + board + eMMC, or 36 TB if you want to be safe against two node failures at the same time.
    
• Broken out UART, 5V, 3.3V, GND
  
• Fan controller that can be steered over I²C or SPI, e.g. to have a daemon on the monitor board receiving temp readings from all the other boards over the network and set the speed that way
  
• Only 1 HDMI port
  
• A way for the monitor board to reset the other boards
  
• A suggestion to put SD cards on the top layer
  
• None of the RPi specific jumpers
  
• No USB 2.0 hub to provide internal USB 2.0 pins (I thought that was a bit on the unnecessary side, but can be added into the specs if that's something people really care about)
  
• A suggestion to have I²C broken out as STEMMA/QWIIC, maybe SPI too, for adding whatever sensing capabilities one might want in a server system (e.g. power monitoring, more temperature monitoring, case open sensing, access to a flash chip, etc.
  

(07-24-2022, 04:42 PM)gadgeteer Wrote: Just wondering if you tried the SOQuartz modules on the Super6C mobo, and if so how it went?  Seems like it would be a lot of fun.

(08-05-2022, 06:00 AM)adamfowleruk Wrote: I can confirm the current soquartz-cm4 manjaro-arm image does find and load the driver for an NVMe device. The u-boot version used by Manjaro Arm doesn't have built in NVMe boot support, but you can load the bootloader fine from an eMMC on the SOQuartz and then this boots from the NVMe drive. I've confirmed you can do the same with the SD Card too.