At long last :-)
Thanks to Tl Lim, my clusterboard prototype for SOPine64 modules arrived a couple of days ago and I started to set it up in an old ITX rack mount case. I will try and post some more thoughts and results during the initial bring up and testing.
The board itself is of mini-ITX formfactor and provides slots for 7 SOPine64 modules networked by the onboard GbE switch. Some interfaces of each module are exposed on the board like two USB ports and a header with UART, SPI, I2C and things like RESET, PWR_ON...
Edit 1: 2017/12/16
My current plan to power the board (5V input only, so maybe 15-20A max) is to use the normal FlexATX PSU I used in the case. The Seasonic SS250-SU is a 250W ATX12V 2.3 unit that provides 17A@5V, 22A@12V and 14A@3,3V. Since the topology of those ATX PSU's doesnt allow for low/no load operation each rail has to be loaded with a minimum load specified in the datasheet. For the SS250-SU this is 3.3V - zero load, 5V - 0.5A, 12V-2A. To not waste the 24W on the 12V I will probably use the DC/DC converter in the pictures and go from 12V to 5V for efficiency reasons...
Next step will probably be to completetly rewire the ATX power cable, fans and front panel connectors with some veroboard to get some controls in and add some features like thermal management, power sequencing and safety... the front panel probably also needs some kind of status display...
Edit 2: 2017/12/17
On the software side I will start using mainline firmware/spl/u-boot right away since the nodes will boot from the SOPine' s on-board SPI flash and pull their working images from the network... have to do some research on the state of those components upstream, but from what I heard everything should be already in place. Getting the SPI images on to the Flash can either be done through FEL mode / USB (WIP) or by booting from a sdcard...
Edit 3: Clusterboard 2.1 schematic
Tl Lim provided me with the latest clusterboard hardware schematics for reference:
PINE64 CLUSTER BOARD_2_1.pdf (Size: 572.46 KB / Downloads: 2,581)
Edit 4: 2017/12/17
I built an Armbian mainline image for a first quick test that boots from sdcard, which seems to work fine, but GbE does not seem to work stable yet, after some changes to the dts I at least got a link... I then tried with ayufan's last xenial-minimal image (legacy kernel) and that worked fine out of the box...
first boot...
two nodes...
I ran some quick iperf3 tests between the nodes and to an external box and got between 800-900Mbit/s , so network hardware seems to be OK... next step is to fix the mainline image as I don't really want to work with legacy kernel here...
I ordered some small 14x14x5mm copper heatsinks and two 80x80mm fans now to get some cooling for the cluster.
Thanks to Tl Lim, my clusterboard prototype for SOPine64 modules arrived a couple of days ago and I started to set it up in an old ITX rack mount case. I will try and post some more thoughts and results during the initial bring up and testing.
The board itself is of mini-ITX formfactor and provides slots for 7 SOPine64 modules networked by the onboard GbE switch. Some interfaces of each module are exposed on the board like two USB ports and a header with UART, SPI, I2C and things like RESET, PWR_ON...
Edit 1: 2017/12/16
My current plan to power the board (5V input only, so maybe 15-20A max) is to use the normal FlexATX PSU I used in the case. The Seasonic SS250-SU is a 250W ATX12V 2.3 unit that provides 17A@5V, 22A@12V and 14A@3,3V. Since the topology of those ATX PSU's doesnt allow for low/no load operation each rail has to be loaded with a minimum load specified in the datasheet. For the SS250-SU this is 3.3V - zero load, 5V - 0.5A, 12V-2A. To not waste the 24W on the 12V I will probably use the DC/DC converter in the pictures and go from 12V to 5V for efficiency reasons...
Next step will probably be to completetly rewire the ATX power cable, fans and front panel connectors with some veroboard to get some controls in and add some features like thermal management, power sequencing and safety... the front panel probably also needs some kind of status display...
Edit 2: 2017/12/17
On the software side I will start using mainline firmware/spl/u-boot right away since the nodes will boot from the SOPine' s on-board SPI flash and pull their working images from the network... have to do some research on the state of those components upstream, but from what I heard everything should be already in place. Getting the SPI images on to the Flash can either be done through FEL mode / USB (WIP) or by booting from a sdcard...
Edit 3: Clusterboard 2.1 schematic
Tl Lim provided me with the latest clusterboard hardware schematics for reference:
PINE64 CLUSTER BOARD_2_1.pdf (Size: 572.46 KB / Downloads: 2,581)
Edit 4: 2017/12/17
I built an Armbian mainline image for a first quick test that boots from sdcard, which seems to work fine, but GbE does not seem to work stable yet, after some changes to the dts I at least got a link... I then tried with ayufan's last xenial-minimal image (legacy kernel) and that worked fine out of the box...
first boot...
two nodes...
I ran some quick iperf3 tests between the nodes and to an external box and got between 800-900Mbit/s , so network hardware seems to be OK... next step is to fix the mainline image as I don't really want to work with legacy kernel here...
I ordered some small 14x14x5mm copper heatsinks and two 80x80mm fans now to get some cooling for the cluster.
Come have a chat in the Pine IRC channel >>
- will post details - presently it's 4 nodes (one of my sopine modules decided to melt itself 
)
