https://forum.pine64.org/ Sat, 16 May 2026 15:43:26 +0000 MyBB https://forum.pine64.org/showthread.php?tid=7384 Mon, 15 Apr 2019 19:31:34 +0000 deckard1138]]> https://forum.pine64.org/showthread.php?tid=7384
I was wondering if picoclusters can be used for cloud hosting/storage servers

Many thanks

deckard1138]]>
I was wondering if picoclusters can be used for cloud hosting/storage servers

Many thanks

deckard1138]]> https://forum.pine64.org/showthread.php?tid=3492 Sat, 18 Feb 2017 15:07:46 +0000 Paraplegic Racehorse]]> https://forum.pine64.org/showthread.php?tid=3492
There once was Kerrighed and OpenSSI, both of which could present the cluster to the user/software as a single system image and hand the transport and messaging layer invisibly. Kerrighed hasn't been active since kernel 2.x and OpenSSI was last updated six or seven years ago.

Is there still a way to create a single system image type cluster?]]>
There once was Kerrighed and OpenSSI, both of which could present the cluster to the user/software as a single system image and hand the transport and messaging layer invisibly. Kerrighed hasn't been active since kernel 2.x and OpenSSI was last updated six or seven years ago.

Is there still a way to create a single system image type cluster?]]> https://forum.pine64.org/showthread.php?tid=2138 Fri, 02 Sep 2016 16:59:30 +0000 kermitas]]> https://forum.pine64.org/showthread.php?tid=2138 https://forum.pine64.org/showthread.php?tid=1909 Mon, 01 Aug 2016 18:56:28 +0000 tllim]]> https://forum.pine64.org/showthread.php?tid=1909 http://climbers.net/sbc/bargain-pine-a64-cluster/

  IMG16_6456_web.JPG (Size: 166.58 KB / Downloads: 902) ]]> http://climbers.net/sbc/bargain-pine-a64-cluster/

  IMG16_6456_web.JPG (Size: 166.58 KB / Downloads: 902) ]]> https://forum.pine64.org/showthread.php?tid=1840 Sun, 24 Jul 2016 15:19:48 +0000 cdslashetc]]> https://forum.pine64.org/showthread.php?tid=1840
It looks like Torque on Ubuntu could be a winner, following these instructions:

https://jabriffa.wordpress.com/2015/02/1...14-04-lts/

update: the head node set up procedure worked perfectly on ubuntu xenial but needed to use real IP in /etc/hosts rather than 127.0.1.1

https://jabriffa.wordpress.com/2015/03/2...14-04-lts/

I also took a stab at Son of Grid Engine on Ubuntu and the gridengine-* packages on Debian.

For Son of Grid Engine, sge-common installed, but I could only find amd64 package for the binaries.

For gridengine-* on Debian, I ran into a similar problem that only armhf binaries were available, so I tried this:

dpkg --add-architecture armhf

but the binaries still would not install due to unresolvable dependency errors.]]>
It looks like Torque on Ubuntu could be a winner, following these instructions:

https://jabriffa.wordpress.com/2015/02/1...14-04-lts/

update: the head node set up procedure worked perfectly on ubuntu xenial but needed to use real IP in /etc/hosts rather than 127.0.1.1

https://jabriffa.wordpress.com/2015/03/2...14-04-lts/

I also took a stab at Son of Grid Engine on Ubuntu and the gridengine-* packages on Debian.

For Son of Grid Engine, sge-common installed, but I could only find amd64 package for the binaries.

For gridengine-* on Debian, I ran into a similar problem that only armhf binaries were available, so I tried this:

dpkg --add-architecture armhf

but the binaries still would not install due to unresolvable dependency errors.]]> https://forum.pine64.org/showthread.php?tid=1522 Sun, 26 Jun 2016 01:52:09 +0000 salmangano]]> https://forum.pine64.org/showthread.php?tid=1522 http://singleboard.org Feedback and suggestions appreciated

  smallfig3.png (Size: 484.94 KB / Downloads: 885) ]]> http://singleboard.org Feedback and suggestions appreciated

  smallfig3.png (Size: 484.94 KB / Downloads: 885) ]]> https://forum.pine64.org/showthread.php?tid=988 Sun, 08 May 2016 00:05:15 +0000 baryluk]]> https://forum.pine64.org/showthread.php?tid=988
I was interested how the Pine64 with its very low price would stand up against other approaches. I am interested mostly in mixed CPU intensive workload. Some floating point, but also some unstructured code.

I have chosen povray, as easily available benchmark, and something that might be actually close to what I would like to test and run on pine64 cluster.

Even if results are negative I might consider pine just for distributed software development and testing. But if the economy is on a bad side, it will not make it very practical replacement for other computing platforms.

I explicitly excluded GPUs, as they are in their specific domain, will deliver best performance, both in absolute terms, and probably in perf/W and perf/$. But development for them isn't that easy and they are not suited for generic codes.

I had an access to one PINE64+ and one PC with relatively modern Intel CPU (Sandy Bridge architecture, i3930K 3.2GHz running at 4.2GHz).

Running Debian testing / unstable.

I compiled povray manually from debian sources (povray 3.7.0), with gcc 6.1 on amd64, and gcc 7.0 (custom built git version from few days ago), on aarch64. This way I got about 15% performance improvement both on Sandy Bridge CPU and on A64 SoC. Debian generic packages in testing/unstable are already compiled with -O3, but without specific -march or -mtune options to tune instruction scheduling, use cache information, etc and such.

(Note that Sandy Bridge lacks AVX and AVX2, but I doubt this would help much in this benchmark. Haswell/Broadwell might bring about 20% of generic IPC improvements, and that is the estimate I used below).

I used FDO and LTO, with final compilation using collected profiles from povray benchmark using -O3 -march=native -ffast-math -fomit-frame-pointers. I tested with other switches, but they do not bring any additional benefits or degrade performance. Most of the performance benefits are from -march=native (especially on Sandy Bridge), just a little from -ffast-math, and FDO/LTO adds few more% of improvement (but also makes binary smaller, helping with the cache utilization).

Results.

I used:
echo | time povray -benchmark

Doing standard pov ray benchmark with 512x512 target output, and adaptive subsampling, for a total of 294912 pixels, ~776k samples, ~2.63 samples/pixel. The benchmark does have very varied structure, with both simple and complex objects,. simple and complex regions, some areas with and without reflection, refraction, aniostropy, complex and simple shading, high and small spatial density of objects, mathematically complex objects and simple ones. For textured objects it uses exclusively procedural textures (including Perlin noise and other fractal methods). It puts small pressure on memory (just few megabytes of memory used at most), and small pressure on memory bandwidth (almost everything fits in the cache, and there is considerably more arithmetic and cpu code than memory accesses, also due to the lack of pregenerated textures / images).

The time below is real time passed on the main Trace pass in povray. (data parsing, data structures creation, photon time, excluded as they are only about 1% of total time, and not all are multithreaded).
 
i7-3930K 3.2GHz @ 4.2GHz, 32nm process, 32GB RAM:  (using 12 threads)

time: 90.227 s  (107.7s on debian generic build)
pixels/s: 3267
power at load: 270W (full system estimate)
power at idle: 100W (full system estimate)
minimal full system price: 450$ (estimate of minimal full system price. 8GB ram, no case, no switch. realistic)
pixels/s/$: 7.26
pixels/s/W: 12.1

Pine64 (sun50iw1p1), 2GB RAM  (with small heatsink): (using 4 threads)

time: 1359 s  (1949 s on debian generic build)
pixels/s: 217  (151.3 on generic build)
power at load: 6W (full system estimate including potential ethernet switch port amortized power)
power at idle: 1.5W (without ethernet)
minimal full system price: 20$ (includes PSU, cabling, heat sink, amortized price of ethernet switch port, but no cases or mounting hardware. optimistic)
pixels/s/$: 10.85
pixels/s/W: 36.17


Speculative estimate for more modern CPU:

i7-5820K 3.3GHz @ 4.2GHz, 6 core, 2x4GB RAM, 22nm process

time: 80 s
pixels/s: 3700
power at load: 250W
power at idle: 60W
minimum full system price: 550$
pixels/s/$: 6.7
pixels/s/W: 14.7


Note that I am only comparing very high end Intel cpus, with 6 cores. Comparison to i5 and i7 4 cores CPU, or AMD CPUs would make it probably considerably in favor on the x86. Both for perf/W and perf/$. I do have an access to few more CPUs, including very low power ones, and would like to update this list shortly.


Summary:

Pine64+ cluster might be realistically speaking viable alternative the x86 high performance computer in some compute intensive workloads. However, it will require about 20 Pine64 boards to match performance of the x86. But with less memory (also distributed across more devices) and memory bandwidth limits, and management + custom software overheads. To fully utilize Pine64 low price and low power usage to compute with other platforms, one needs to bring the costs as low as possible (every 1$ counts), but using shared PSU, and cheap cables and cooling solutions and DIY mounting options. Just making it cost more than 30$ will make it economically impractical. 25$ might be ok (19$ for PINE, and 6$ for different parts), 20$ would be the best.

If matched with initial cost, PINE64 might be very attractive option in the long run, due to very low idle power usage, and very high performance/W metrics.

Future work:

More tests. Calculate 2-year ownership cost at 20% utilization (20% of the time system fully loaded, 80% idle).

What do you think? Any other comparisons to make? Maybe things like webserving? memory based caching? storage of some sort?

I would love to see somebody build ~50 nodes cluster with price, performance and power details.]]>
I was interested how the Pine64 with its very low price would stand up against other approaches. I am interested mostly in mixed CPU intensive workload. Some floating point, but also some unstructured code.

I have chosen povray, as easily available benchmark, and something that might be actually close to what I would like to test and run on pine64 cluster.

Even if results are negative I might consider pine just for distributed software development and testing. But if the economy is on a bad side, it will not make it very practical replacement for other computing platforms.

I explicitly excluded GPUs, as they are in their specific domain, will deliver best performance, both in absolute terms, and probably in perf/W and perf/$. But development for them isn't that easy and they are not suited for generic codes.

I had an access to one PINE64+ and one PC with relatively modern Intel CPU (Sandy Bridge architecture, i3930K 3.2GHz running at 4.2GHz).

Running Debian testing / unstable.

I compiled povray manually from debian sources (povray 3.7.0), with gcc 6.1 on amd64, and gcc 7.0 (custom built git version from few days ago), on aarch64. This way I got about 15% performance improvement both on Sandy Bridge CPU and on A64 SoC. Debian generic packages in testing/unstable are already compiled with -O3, but without specific -march or -mtune options to tune instruction scheduling, use cache information, etc and such.

(Note that Sandy Bridge lacks AVX and AVX2, but I doubt this would help much in this benchmark. Haswell/Broadwell might bring about 20% of generic IPC improvements, and that is the estimate I used below).

I used FDO and LTO, with final compilation using collected profiles from povray benchmark using -O3 -march=native -ffast-math -fomit-frame-pointers. I tested with other switches, but they do not bring any additional benefits or degrade performance. Most of the performance benefits are from -march=native (especially on Sandy Bridge), just a little from -ffast-math, and FDO/LTO adds few more% of improvement (but also makes binary smaller, helping with the cache utilization).

Results.

I used:
echo | time povray -benchmark

Doing standard pov ray benchmark with 512x512 target output, and adaptive subsampling, for a total of 294912 pixels, ~776k samples, ~2.63 samples/pixel. The benchmark does have very varied structure, with both simple and complex objects,. simple and complex regions, some areas with and without reflection, refraction, aniostropy, complex and simple shading, high and small spatial density of objects, mathematically complex objects and simple ones. For textured objects it uses exclusively procedural textures (including Perlin noise and other fractal methods). It puts small pressure on memory (just few megabytes of memory used at most), and small pressure on memory bandwidth (almost everything fits in the cache, and there is considerably more arithmetic and cpu code than memory accesses, also due to the lack of pregenerated textures / images).

The time below is real time passed on the main Trace pass in povray. (data parsing, data structures creation, photon time, excluded as they are only about 1% of total time, and not all are multithreaded).
 
i7-3930K 3.2GHz @ 4.2GHz, 32nm process, 32GB RAM:  (using 12 threads)

time: 90.227 s  (107.7s on debian generic build)
pixels/s: 3267
power at load: 270W (full system estimate)
power at idle: 100W (full system estimate)
minimal full system price: 450$ (estimate of minimal full system price. 8GB ram, no case, no switch. realistic)
pixels/s/$: 7.26
pixels/s/W: 12.1

Pine64 (sun50iw1p1), 2GB RAM  (with small heatsink): (using 4 threads)

time: 1359 s  (1949 s on debian generic build)
pixels/s: 217  (151.3 on generic build)
power at load: 6W (full system estimate including potential ethernet switch port amortized power)
power at idle: 1.5W (without ethernet)
minimal full system price: 20$ (includes PSU, cabling, heat sink, amortized price of ethernet switch port, but no cases or mounting hardware. optimistic)
pixels/s/$: 10.85
pixels/s/W: 36.17


Speculative estimate for more modern CPU:

i7-5820K 3.3GHz @ 4.2GHz, 6 core, 2x4GB RAM, 22nm process

time: 80 s
pixels/s: 3700
power at load: 250W
power at idle: 60W
minimum full system price: 550$
pixels/s/$: 6.7
pixels/s/W: 14.7


Note that I am only comparing very high end Intel cpus, with 6 cores. Comparison to i5 and i7 4 cores CPU, or AMD CPUs would make it probably considerably in favor on the x86. Both for perf/W and perf/$. I do have an access to few more CPUs, including very low power ones, and would like to update this list shortly.


Summary:

Pine64+ cluster might be realistically speaking viable alternative the x86 high performance computer in some compute intensive workloads. However, it will require about 20 Pine64 boards to match performance of the x86. But with less memory (also distributed across more devices) and memory bandwidth limits, and management + custom software overheads. To fully utilize Pine64 low price and low power usage to compute with other platforms, one needs to bring the costs as low as possible (every 1$ counts), but using shared PSU, and cheap cables and cooling solutions and DIY mounting options. Just making it cost more than 30$ will make it economically impractical. 25$ might be ok (19$ for PINE, and 6$ for different parts), 20$ would be the best.

If matched with initial cost, PINE64 might be very attractive option in the long run, due to very low idle power usage, and very high performance/W metrics.

Future work:

More tests. Calculate 2-year ownership cost at 20% utilization (20% of the time system fully loaded, 80% idle).

What do you think? Any other comparisons to make? Maybe things like webserving? memory based caching? storage of some sort?

I would love to see somebody build ~50 nodes cluster with price, performance and power details.]]> https://forum.pine64.org/showthread.php?tid=820 Thu, 28 Apr 2016 14:40:20 +0000 jproffer]]> https://forum.pine64.org/showthread.php?tid=820
  IMG_0848.JPG (Size: 419.22 KB / Downloads: 1190)
Currently powered by a homemade USB power hub (17 port) and a computer PSU. Re-doing the USB hub to a cleaner implementation, 2 8-port cards

  IMG_0849.JPG (Size: 351.3 KB / Downloads: 1047) ]]>
  IMG_0848.JPG (Size: 419.22 KB / Downloads: 1190)
Currently powered by a homemade USB power hub (17 port) and a computer PSU. Re-doing the USB hub to a cleaner implementation, 2 8-port cards

  IMG_0849.JPG (Size: 351.3 KB / Downloads: 1047) ]]> https://forum.pine64.org/showthread.php?tid=427 Sun, 13 Mar 2016 07:03:17 +0000 Madroxprime]]> https://forum.pine64.org/showthread.php?tid=427   If you use Longsleep's Ubuntu image, you can install either of the MPI libraries(OpenMPI or MPICH) from the repositories if you're not keen on compiling them from source. They both seem to be the most recent versions, so that's neat.  You might need to edit sources.list to include the universe repositories.
  I'll post details as I get more into it. Right now I've only got the one Pine so I won't be able to get much information out besides what you need and how to get it.]]>   If you use Longsleep's Ubuntu image, you can install either of the MPI libraries(OpenMPI or MPICH) from the repositories if you're not keen on compiling them from source. They both seem to be the most recent versions, so that's neat.  You might need to edit sources.list to include the universe repositories.
  I'll post details as I get more into it. Right now I've only got the one Pine so I won't be able to get much information out besides what you need and how to get it.]]> https://forum.pine64.org/showthread.php?tid=238 Mon, 01 Feb 2016 22:36:04 +0000 T_ech]]> https://forum.pine64.org/showthread.php?tid=238
IK that is a lot to think about and I am sorry.

P.S. How expensive would the cluster cost? ]]>
IK that is a lot to think about and I am sorry.

P.S. How expensive would the cluster cost? ]]> https://forum.pine64.org/showthread.php?tid=229 Sun, 31 Jan 2016 01:11:51 +0000 Crispin]]> https://forum.pine64.org/showthread.php?tid=229
I am intrigued by the partnership with PicoCluster and what this might enable for end-users in our homes.

Does anyone have any ideas/plans on what clustering PINE64s might enable? Or does it simply make one's PINE64 unit more powerful/fast, as opposed to adding more features?

]]>
I am intrigued by the partnership with PicoCluster and what this might enable for end-users in our homes.

Does anyone have any ideas/plans on what clustering PINE64s might enable? Or does it simply make one's PINE64 unit more powerful/fast, as opposed to adding more features?

]]> https://forum.pine64.org/showthread.php?tid=195 Mon, 25 Jan 2016 09:24:33 +0000 W1SPY]]> https://forum.pine64.org/showthread.php?tid=195
I know it's ambitious being as though it hasnt even shipped yet, gotta dream big.]]>
I know it's ambitious being as though it hasnt even shipped yet, gotta dream big.]]> https://forum.pine64.org/showthread.php?tid=124 Fri, 15 Jan 2016 11:20:42 +0000 Ron Piggott]]> https://forum.pine64.org/showthread.php?tid=124