DeskPi Super6C CM4 board with the SOQuartz
#1
Hello all,

I've recently taken delivery of a DeskPi Super6C motherboard that can house 6x RPi CM4 on one side, and 6x NVMe SSD on the other side: https://deskpi.com/collections/deskpi-su...-supported

The CM4s are still hard to come by, and so I was thinking about trying to get one of the SOQuartz integrated with this mini-ITX sized host board. Before I buy a bunch of SOQuartz though I wanted to ask if the community thought this a crazy idea or not?

I'm very happy hacking around with Manjaro (I use it as a daily driver on my personal machine), and using serial access to machines, playing around with dts files, kernels, and so on for embedded, so don't think there's a huge issue there for me. I note that a few people are collaborating on SOQuartz on RPi CM4 host boards already: https://github.com/geerlingguy/raspberry...issues/336

Does anyone have any strong feelings on my above pairing at all? Is it a worthwhile project?

(Background: I'm looking to build a relatively low power multi-node cluster for lightweight Ubuntu (and Ceph) eventually with BYOH Kubernetes on top.)

Thanks in advance for all opinions and advice.
  Reply
#2
(06-25-2022, 12:21 PM)adamfowleruk Wrote: Hello all,

I've recently taken delivery of a DeskPi Super6C motherboard that can house 6x RPi CM4 on one side, and 6x NVMe SSD on the other side: https://deskpi.com/collections/deskpi-su...-supported

The CM4s are still hard to come by, and so I was thinking about trying to get one of the SOQuartz integrated with this mini-ITX sized host board. Before I buy a bunch of SOQuartz though I wanted to ask if the community thought this a crazy idea or not?

I'm very happy hacking around with Manjaro (I use it as a daily driver on my personal machine), and using serial access to machines, playing around with dts files, kernels, and so on for embedded, so don't think there's a huge issue there for me. I note that a few people are collaborating on SOQuartz on RPi CM4 host boards already: https://github.com/geerlingguy/raspberry...issues/336

Does anyone have any strong feelings on my above pairing at all? Is it a worthwhile project?

(Background: I'm looking to build a relatively low power multi-node cluster for lightweight Ubuntu (and Ceph) eventually with BYOH Kubernetes on top.)

Thanks in advance for all opinions and advice.

Hey Adam,

Just wondering if you tried the SOQuartz modules on the Super6C mobo, and if so how it went?  Seems like it would be a lot of fun.

Thanks!
  Reply
#3
This is an interesting board, I note a few things though that make it less ideal for SOQuartz than for RPi:
  • no UART, which is very unfortunate since you'll most likely need this in some capacity
  • SD card on the bottom, which means its harder to access, which you have to do more on the SOQuartz unless you want to bother with Rockchip's USB download mode that I don't think we've documented on the wiki anywhere

It inspired me to draft up specifications for an "ideal" (but within the realm of possible) SOQuartz mini-ITX carrier board of my own though, which I'll leave here for comments: https://wiki.pine64.org/wiki/User:Counte...rier_Board

Highlights of suggested differences vs. the Super6C:
  • SATA for each board (removes the USB 3.0 capability from the monitor board)
    • Allows to use this as a distributed (node redundancy rather than block device redundancy) storage cluster (Ceph, GlusterFS) with an HDD cached by an SSD. Imagine 6 SOQuartzes, each with a 18 TB drive (like $315 each right now) and a 500GB TLC NVMe SSD (like $50 now) for caching with bcache. At 2x redundancy, that leaves you with 54 TB of SSD-cached storage for like $2190 for the storage and around $600 for SOQuartz + board + eMMC, or 36 TB if you want to be safe against two node failures at the same time.
  • Broken out UART, 5V, 3.3V, GND
  • Fan controller that can be steered over I²C or SPI, e.g. to have a daemon on the monitor board receiving temp readings from all the other boards over the network and set the speed that way
  • Only 1 HDMI port
  • A way for the monitor board to reset the other boards
  • A suggestion to put SD cards on the top layer
  • None of the RPi specific jumpers
  • No USB 2.0 hub to provide internal USB 2.0 pins (I thought that was a bit on the unnecessary side, but can be added into the specs if that's something people really care about)
  • A suggestion to have I²C broken out as STEMMA/QWIIC, maybe SPI too, for adding whatever sensing capabilities one might want in a server system (e.g. power monitoring, more temperature monitoring, case open sensing, access to a flash chip, etc.

This is obviously not a real product whether in development or not but a specification, but it's possible within the constraints of the interface, hardware and desired price point, so it's a bit beyond just playing ideas guy. I'll see if PINE64 itself is interested in picking this up or if the SOQuartz Blade covers this market for them already.

Occasional Linux Kernel Contributor, Avid Wiki Updater, Ask Me About Quartz64
Open Hardware Quartz64 Model A TOSLink Adapter
Pi-bus GPIO Extender For ROCKPro64 And Quartz64
  Reply
#4
(07-24-2022, 04:42 PM)gadgeteer Wrote: Just wondering if you tried the SOQuartz modules on the Super6C mobo, and if so how it went?  Seems like it would be a lot of fun.

Hi there! Yes I have now got it installed. I've managed to boot Manjaro Arm using the 20220718 build of the soquartz dev images. Some things are not working, but the OTG USB (unpowered - you need a powered OTG hub!) and Ethernet and HDMI do work. I've now nearly snapped that USB OTG connector off though, so I'm busy now with a new DTS file for the DeskPi Super6C carrier board and SOQuartz combination specifically so I can use the 2 USB 2.0 Type A ports on the carrier board that are only connected to Node 1.

Here's the binary link. pgwipeout has done all of the work so far with the CM4IO board, so I'm basically building on top of that to support the Super6C carrier board. I'm using the xfce version: https://github.com/manjaro-arm/soquartz-...g/20220718

I'm jotting down my notes on this combination here: https://github.com/adamfowleruk/deskpi-super6c

And doing a couple of YouTube videos on it. Only an intro there currently in the 'July 2022 update' video under the 'Edge' Playlist: https://www.youtube.com/c/adamfowleruk - My next video coming out on Mon 8th Aug will be a DeskPi super6c overview though!

I can confirm the current soquartz-cm4 manjaro-arm image does find and load the driver for an NVMe device. The u-boot version used by Manjaro Arm doesn't have built in NVMe boot support, but you can load the bootloader fine from an eMMC on the SOQuartz and then this boots from the NVMe drive. I've confirmed you can do the same with the SD Card too.

The carrier board appears to have some sort of extra microcontroller and serial chip. I've not investigated that yet. I want a solid boot build so others can try first at which point I'll order some more SOQuartz boards and build a cluster. I'm thinking microk8s should work well here. (Or the VMware ESXi arm fling if I get adventurous - I work for VMware).
  Reply
#5
(08-05-2022, 06:00 AM)adamfowleruk Wrote: I can confirm the current soquartz-cm4 manjaro-arm image does find and load the driver for an NVMe device. The u-boot version used by Manjaro Arm doesn't have built in NVMe boot support, but you can load the bootloader fine from an eMMC on the SOQuartz and then this boots from the NVMe drive. I've confirmed you can do the same with the SD Card too.

Correct, the u-boot versions we've got (both the mainline based and the downstream one) don't have PCIe support at the moment. Even if they did though, the board would still have to load u-boot from either SD or eMMC, since SD/eMMC/SPI are the things the maskrom (lowest level early boot code burned into chip) can load the bootloader from.

So since I don't think there's SPI flash on the SOQuartz modules, you're already gonna be using either SD or eMMC, at which point putting the kernel onto that as well isn't that big of a deal.

Occasional Linux Kernel Contributor, Avid Wiki Updater, Ask Me About Quartz64
Open Hardware Quartz64 Model A TOSLink Adapter
Pi-bus GPIO Extender For ROCKPro64 And Quartz64
  Reply
#6
I just got my modules for the DeskPi Super6C today and am experimenting with it. One notable thing I ran into is being unable to successfully reboot. It shuts down correctly AFAICT but then never comes back up. I need to either long press the power button on the Super6C or unplug and plug the power to the board to get it to reboot again.

Would be nice to know that this is also the case with your setup @adamfowleruk

Another thing I'm struggling with currently is to get the nvme drive to work. I get the following output in dmesg (filtered by "nvme"):


Code:
[    0.603723] nvme nvme0: pci function 0000:01:00.0
[    0.603824] nvme 0000:01:00.0: enabling device (0000 -> 0002)
[    0.629624] nvme nvme0: allocated 64 MiB host memory buffer.
[    0.639069] nvme nvme0: 4/0/0 default/read/poll queues
[   31.896584] nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS read failed (134)
[   34.044873] nvme0n1: I/O Cmd(0x2) @ LBA 488396928, 8 blocks, I/O Error (sct 0x3 / sc 0x71)
[   34.044887] I/O error, dev nvme0n1, sector 488396928 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[   34.095180] nvme 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible
[   34.146540] nvme nvme0: Removing after probe failure status: -19
[   34.195357] Buffer I/O error on dev nvme0n1, logical block 61049616, async page read
[   34.195420] nvme0n1: detected capacity change from 488397168 to 0
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  SOQuartz ATX Baseboard nixcamic 0 512 02-05-2022, 11:55 PM
Last Post: nixcamic
  Lamenting the fact that only 2 lanes of LVDS are exposed on SOQuartz evankrall 0 528 12-13-2021, 01:07 PM
Last Post: evankrall

Forum Jump:


Users browsing this thread: 2 Guest(s)