pdeclib pilib modules for Pine, Rock, Pinebook, Sopine

[MarkHaysHarris777]

In March of 2014 I wrote a transcendental functions module for python ( in pure python ) which leverages the relatively new Decimal module; especially fast in Python3 on x86 machines.  

As it turns out these routines are also especially fast in Python3 on the Rock64 and the Pinebook;  although, I have used them to benchmark on the PineA64 also;  fast in its own right. 

The Decimal module is written and optimized in C.  My module(s) live on pypi and are being made available here for others who may want to experiment with big-nums maths or who may specifically want to run my PI benchmarks themselves. The pdeclib pilib module(s) will run on python2 or python3 unchanged;  they may be run in the REPL in "calculator" mode or they may be easily scripted.  The module(s) provide transcendental functions to arbitrary precision as well provide several PI routines which converge relatively quickly -- particularly fast is   piagm2().  These routines are also useful for those who want to study maths codes for convergent series. 

The pdeclib package may be obtained here:

The package contains pdeclib.py (the main transcendental functions module) and pilib.py (several useful historical PI routines).  I hope they are useful, but recognize that they may not be complete, nor free from bugs;  however, they are intended to be used and studied, and they have been available on-line for several years.

To use the modules simply place the two files in a working directory (or in your PythonPath) and then run the python REPL of choice in that directory , or path.  Then import the modules in the normal way:

>>>  from pilib import *

The module pilib will import pdeclib;  although pdeclib may be imported alone.  The default number of digits used is 42.  This may be changed with the function:

>>>  dscale(1010)

The above will set the number of big-num Decimal places to 1,010.  To compute PI run:

>>>  piagm2()

Read through the source (header and readme files) for caveat(s) and to find the syntax of the various functions provided by the package. There are no blobs, and the source is 100% pure python code;  what you see is what you get.

The following script is a Python3 example script for running the PI benchmark on any of the Pine boards and a gnu+linux image.  Its a simple script which starts the Python interpreter, imports the requisite modules, and executes the piagm2()  AGM PI routine, to 1010 places.  This result will be accurate to +|-  in the last two places.  The signature digits for the 1000th place are  164201989.  Also, at position 762 you will find six nines ... 999999.

PI-calc.sh

Code:

#!/usr/bin/python3
from pilib import *
dscale(1010)
print(piagm2())

To run the codes above use the 'time' command with the script and the modules in your working directory:

time  ./PI-calc.sh

Don't forget to make the script executable with :  chmod  0754  PI-calc.sh

The above command will give you 1000+ correct digits of PI in a few hundred milliseconds, and will also tell you how much 'real' ,  'system' , and 'user'  time was used.  You might find it interesting to compare the computations on our four platform boards for  1K, 5K, and 10K  digits of PI !

These little SoC(s) particularly running aarch64 ArmV8 64bits are very fast;  so, knock your SoC(s) off !

Note:  It is interesting to run four or more terminals [ each running piagm2() ]  to load the processor (all cores) for watching the temps in /sys/class/thermal/  or for experimenting with SMP.  

Note:  Other things affect the speed of these calcs,  like gcc version  and kernel version;  experiment with different gnu+linux builds;  try these codes and compare with different images.   Hav'fun.

Note:  Some believe this kind of bench-marking to be silly;  to the contrary,  this is a tool that provides valuable insight into the comparative operation of the SoC(s) for maths, as well for 32bit vs 64bit, as well for cache vs RAM.  Its just one tool and only gives a partial picture, yet is a fun way to explore your SoC(s), and learn some maths and python at the same time !

Note:  here is sample run from rock64 xenial minimal image 0.3.7:

Code:

rock64@rock64:~/Python$ time ./PI-calc.sh
3.141592653589793238462643383279502884197169399375105820974944592307816406286208
99862803482534211706798214808651328230664709384460955058223172535940812848111745
02841027019385211055596446229489549303819644288109756659334461284756482337867831
65271201909145648566923460348610454326648213393607260249141273724587006606315588
17488152092096282925409171536436789259036001133053054882046652138414695194151160
94330572703657595919530921861173819326117931051185480744623799627495673518857527
24891227938183011949129833673362440656643086021394946395224737190702179860943702
77053921717629317675238467481846766940513200056812714526356082778577134275778960
91736371787214684409012249534301465495853710507922796892589235420199561121290219
60864034418159813629774771309960518707211349999998372978049951059731732816096318
59502445945534690830264252230825334468503526193118817101000313783875288658753320
83814206171776691473035982534904287554687311595628638823537875937519577818577805
321712268066130019278766111959092164201989380952572

real    0m0.263s
user    0m0.231s
sys     0m0.029s

Note:  the signature digits for the 5,000th digit are : 132604721

Note:  the signature digits for the 10,000th digit are : 256375678

[MarkHaysHarris777]

Greetings again;  got some more PI fun for you that will literally blow your SoC(s) off using nothing but the Python interpreter and the following two very short scripts -- using integer maths only:

pipipi.py

Code:

digits = 10010
pi = term = 3 * 10 ** (digits+9)

m = 2
while term:
 term = term * ((m-1) * (m-1)) // (4 * m * (m+1))
 pi += term
 m += 2

print(pi // 10 ** 9)

PI-calc2.sh

Code:

#!/usr/bin/python
from pipipi import *

Place these two scripts in your working directory;  be sure to make the PI-calc2.sh script executable with:

chmod 0754 PI-calc2.sh

To run the integer PI calculation use:

time ./PI-calc2.sh


On the Rock64 board this PI calculation will churn out 10,000+ digits of PI in just over 1/2 second.  You can experiment between the Python2 and Python3 interpreter times;  or more fun compare the times on the Pinebook vs PineA64+ vs Rock64 !

Because this small routine uses integer maths to get the digits of a never ending transcendental number, obviously the decimal point is going to be missing !  ... but all the digits will be there;  experiment by increasing the number of digits in the pipipi.py routine.  Its truly amazing to see the power of a tiny computer being used to actually "compute" in that sense of the word.

Note:   The Pinebook calculates 10K digits of PI using integer maths on the Python3 interpreter in 3/4 second !

---

---

---

To put this is perspective for the year 2017 the first computer to accurately  calculate PI to 10K digits occurred in 1958 (when I was three years old);  ... in 1 hour and 40 minutes !

"... in July 1958, an IBM 704 at the Paris Data Processing Center was programmed according to a combination of Machin's formula and the Gregory series;  it yielded 10,000 decimal places in 1 hour and 40 minutes." 

"A year later, in July 1959, the same program was used on an IBM 704 at the Commissarit à l'Energie Atomique in Paris, and 16,167 places were obtained in 4.3 hours."

I reproduced the above precision using integer math on the Rock64 board (16177) and arrived at the result in a blazing 1.483 seconds on the Python2 interpreter ; running on the xenial minimal image 0.3.7 !

   


Quotes taken from :

Beckmann, Petr; A History of PI. The Golem Press (NewYork, 1971)

---

---

[MarkHaysHarris777]

Greetings;

In 1961 Daniel Shanks and John W. Wrench Jr. calculated PI to 100,256 decimal places on an IBM 7090 at the IBM Data Processing Center in New York (free of charge) !

Their paper is fascinating. 


The IBM 7090 is the transistorized version of the  vacuum tube IBM 709.  Shanks and Wrench were able to crank out over 100K digits of PI in a whopping 8 hours and 43 minutes;  using the faster 7090, and by implementing some programming tricks very new at the time.  

Using the Rock64 board and the piagm2() Decimal function I was able to replicate Shanks and Wrench work in just 2 minutes 30.2 seconds; without throttling using only passive cooling (14mmX14mm heatsink).  The integer maths routine completed to 100K digits in a blazing 1 minute 12.5 seconds !

Their paper notes that at that time (September 1961) the limit of machine speed and available ¨ferrite core¨ memory made it quite impossible to calculate 1 million digits ( a simple feat today on any personal computer ) !

Interesting to note that the comparison of the output of the two routines on two different machines ( in two different periods of time) produced the same result;  matching the result of Shanks and Wrench 56 years ago...

[MarkHaysHarris777]

Greetings;  well, you knew this was coming !  

In 1973, one year before I graduated from high school, Jean Guilloud and Martin Bouyer calculated 1,000,000+ digits of PI on a CDC 7600 in 23.3 hours !  For those of you who need reminding, this was the Seymour Cray-designed supercomputer which was Control Data´s competition against the IBM 360 model 195.  This occurred a mere 44 years ago at the end of the Apollo era;  most of you were probably not even alive then.

I have replicated Guilloud and Bouyerś work on both the Rock64 and PineA64 supercomputers in less than 1/2 hour!  

To accomplish this we need to make some adjustments to our Python script which is listed below;  very short and sweet :

pi-fast-calc.py

Code:

import sys
digits = int(sys.argv[1]) if len(sys.argv) > 1 else 100000

def mpz(n): return n
#from gmpy2 import mpz

def atan(pp, qq):
   def bs(a, b):
       if b == a + 1:
           if a == 0: return (1, 1, 1)
           p = mpz(1-2*a)
           q = mpz((2*a+1) * qq * qq)
           return (p, q, p)
       m = (a+b) // 2
       (Pa, Qa, Ta) = bs(a, m)
       (Pb, Qb, Tb) = bs(m, b)
       return (Pa*Pb, Qa*Qb, Pa*Tb + Qb*Ta)
   from math import log10
   (p, q, t) = bs(0, int((digits+9) / log10(qq*qq)))
   return 4 * pp * mpz(10) ** (digits+9) * t // q // qq

pi = atan(44, 57)
pi += atan(7, 239)
pi += atan(-12, 682)
pi += atan(24, 12943)
print(pi // 10**9)

To run the codes above use:

time python pi-fast-calc.py 1000100 >PI_1000_K.txt

I scrubbed the output slightly by removing the first digit ¨3¨ and truncating the number at precisely 1,000,000 digits after the decimal point.  The file size is therefore 1000001 including the line end or file end marker.  The signature for the millionth digit after the decimal point is 779458151 

The times as well the md5 and sha checksums have been included;  and I have appended the one million digits obtained from the Rock64 & PineA64.

Code:

user    21m34.615s  
rock64   PI_1000_K.txt  f776c92eb9b3865128f39e2a096497e9  md5

user    24m50.170s  
pineA64  PI-1000-K.txt  51ecf7f4a842bbcef9c5f3a7bbc9022482ef4d89  sha

The Rock64 runs slightly faster, as well the Rock64´s LPDDR3 memory is also slightly faster;  consequently the Rock64 beat the PineA64 by about 3 minutes sixteen seconds. You can compare your digits with the appended file, or you can run md5sum or shasum against the actual digits to see if your calculation is similar.

To get the digits run tar against the tarball appended below:  first rename the file from PI_1000_K.bmp  to  PI_1000_K.tar.xz  !  (this trick is needed because the forum hates tarballs)

tar  -xzvf  PI_1000_K.tar.xz

To get the md5sum , or shasum , run :

md5sum PI_1000_K.txt

shasum  PI_1000_K.txt 



Hint:  To download the attachment right click and select ¨save link as...¨

[MarkHaysHarris777]

So what ?  What does this tell us, and why is it important ?

Greetings,

In this segment I am providing a utility that will count and analyze the digits of our pi_digits file;  the C codes are posted below with instructions for use.

Aside from the historically valid reasons in computer science for calculating PI to many many digits of accuracy, there are three primary reasons why we might want to engage in this activity with our SoC(s);  but first, what this does not tell us ...

If the calculation(s) were very slow, or if they were not accurate, the test would tell us nothing about the nature of the latency, nor the reason for the failure;  this is a both|and -- if the calculation is both fast and accurate (the definition of efficient performance)  then we can say a great deal about the nature of the system being used to perform the calculation;  such a system would be virtually flawless in anything performed as "computable" and such a system could also be considered absolutely reliable in terms of operation having completed hundreds of billions of operations in a very short period of time without failure!

The primary reasons for performing PI calculations in modern SoC(s) are the following:

1) demonstrate normalized system performance 
2) demonstrate accurate system operation
3) demonstrate normalized output for statistical verification

The first two are most important.  The conjecture for normalization of digits in large numbers of PI digits has the benefit (over a very large calculation) of providing a normalized set of operations (hundreds of billions of them) such that if the calculation is both accurate and fast we can say (taking everything into account: cache, bus, ram, pipe-lining, SoC, clock, OS, and user space) that the "system" is highly efficient. If we are comparing two (or more) SoC(s), without having to know anything about the specifics of any one component of the system necessarily, if both SoC(s) are relatively similar in terms of the test, then we can reasonably compare the two SoC(s) and rank them.

Both of our test SoC(s) are relatively similar;  they complete the test quickly and accurately and both within less than 25 minutes;  yes, one is 3 minutes 16 seconds faster, but the difference is not in orders of magnitude -- not ten times as fast, nor even twice as fast.  However, we have demonstrated that the Rock64 is a more efficient SoC (board and system) in a normalized way.

To test the normalized conjecture (which has not been mathematically proven in a rigorous way) that the digits of PI are in fact highly normalized we can run the following utility which will count and analyze the digits.

pi_digit_dist.c

Code:

// pi_digit_dist.c   v0.6
//
// Mark H. Harris
// 08-10-2017  v0.2  initial utility
//
// changelog
//   v0.3     added standard deviation
//   v0.4     added formatting
//            added minimal error checking
//            fixed for loop inits
//                  no longer requires -std=c99
//   v0.5     adjusted min max
//                     min is first minimum
//                     max is last maximum
//            adjusted formatting
//   v0.6     added Sqrs to stats
//

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

#define  CONST_MOD  48
#define  FILE_OPEN  -1
#define  PARM_USE   -2

int main(int argc, char** argv) {
//  Definitions
    int rc = 0, max=0, min=0;
    FILE* pi_digits;
    char* pi_file;
    int c, i, n, digits, seek_pos;
    int counters[10] = {0};
    float dsum=0, dmean=0, dsqrs=0, sdev, ssdev, dvar;
   double ssqrs=0;

//  Parse parameters argument
    if (argc==4) {
        printf("%s\n\n", argv[0]);
        digits = atoi(argv[2]);
        pi_file = argv[1];
        seek_pos = atoi(argv[3]);
        }
     else {
        printf("\nUsage:\n\npi_digit_dist <pi_digits_filename> <sample_no> <seek_offset>\n\n");
        return PARM_USE;
        }

//  Open the digits file
    pi_digits = fopen(pi_file, "r");
    if (pi_digits != NULL)
        fseek(pi_digits, seek_pos, SEEK_SET);
    else {
        printf("Error: could not open file %s\n\n", argv[1]);
        return FILE_OPEN;
        }

//  Count the unique digits
    for(i=0; i<digits; i++) {
        c = fgetc(pi_digits);
        n = c - CONST_MOD;
        counters[n] += 1;
        }

//  Print the unique digits
    for (i=0; i<10; i++) {
        printf("        %d = %d\n", i,counters[i]);
        if (counters[i]<counters[min])
            min = i;
        if (counters[i]>=counters[max])
            max = i;
        dsum += counters[i];
        }

//  Calculate Standard Deviation
    dmean = dsum / 10.0;
    for (i=0; i<10; i++) {
        dsqrs += pow( (float) counters[i] - dmean, 2);
        ssqrs += pow( (double) counters[i], 2);
        }
    dvar = dsqrs / 9.0;           // sample variance
    ssdev = sqrt(dvar);           // sample s deviation
    sdev = sqrt(dsqrs / 10.0);    // population σ deviation

//  Print statistics
    printf("\n");
    printf("      max = %d (%d)\n", max,counters[max]);
    printf("      min = %d (%d)\n", min,counters[min]);
    printf("\n");
    printf("      sum = %14.3f   Σx\n", dsum);
    printf("     mean = %14.3f   μ\n", dmean);
    printf("     sqrs = %14.3f   Σ(x-μ)²\n", dsqrs);
    printf("     Sqrs = %14.3f   Σ(x)²\n", ssqrs);
    printf(" variance = %14.3f   Σ(x-μ)²/(n-1)\n", dvar);
    printf("        σ = %14.3f   √(Σ(x-μ)²/n)\n", sdev);
    printf("        s = %14.3f   √(Σ(x-μ)²/(n-1))\n", ssdev);

//  Close the digits file
    fclose(pi_digits);
    printf("\n");
    return rc;
}

To compile the code:

gcc -Wall -o pi_digit_dist pi_digit_dist.c -lm

( the -lm flag is necessary to link in the math.h library )


To run the code:

./pi_digit_dist PI_1000_K.txt 7000 700000

In the sample above the utility will count the unique occurrences of the digits in the PI file -- it will begin at offset 700000 in the file and count through 7000 digits.  A sample run is provided below:

Code:

rock64@rock64:~/Python/PI_million$ ./pi_digit_dist pi_1000k_out 7000 700000
./pi_digit_dist

        0 = 708
        1 = 728
        2 = 739
        3 = 653
        4 = 645
        5 = 727
        6 = 742
        7 = 679
        8 = 688
        9 = 691

      max = 6 (742)
      min = 4 (645)

      sum =    7000.0000   Σx
     mean =     700.0000   μ
     sqrs =   10762.0000   Σ(x-μ)²
 variance =    1195.7778   Σ(x-μ)²/(n-1)
        σ =      32.8055   √(Σ(x-μ)²/n)
        s =      34.5800   √(Σ(x-μ)²/(n-1))

  

We could also look through the entire file as well:

Code:

rock64@rock64:~/Python/PI_million$ ./pi_digit_dist pi_1000k_out 1000000 0
./pi_digit_dist

        0 = 99959
        1 = 99758
        2 = 100026
        3 = 100229
        4 = 100230
        5 = 100359
        6 = 99548
        7 = 99800
        8 = 99985
        9 = 100106

      max = 5 (100359)
      min = 6 (99548)

      sum = 1000000.0000   Σx
     mean =  100000.0000   μ
     sqrs =  550908.0000   Σ(x-μ)²
 variance =   61212.0000   Σ(x-μ)²/(n-1)
        σ =     234.7143   √(Σ(x-μ)²/n)
        s =     247.4106   √(Σ(x-μ)²/(n-1))

These statistics also verify the content of the file indirectly as well;  because the digits of PI do not vary,  the statistics regarding the normalized distribution of PI digits will also not change

[MarkHaysHarris777]

---

Here is another sample run :

Code:

pine64@pinebook:~/Python/PI_million$ ./pi_digit_dist pi_1000k_out 777 37699
./pi_digit_dist

        0 = 84
        1 = 84
        2 = 77
        3 = 69
        4 = 80
        5 = 68
        6 = 74
        7 = 72
        8 = 92
        9 = 77

      max = 8 (92)
      min = 5 (68)

      sum =        777.000   Σx
     mean =         77.700   μ
     sqrs =        506.100   Σ(x-μ)²
     Sqrs =      60879.000   Σ(x)²
 variance =         56.233   Σ(x-μ)²/(n-1)
        σ =          7.114   √(Σ(x-μ)²/n)
        s =          7.499   √(Σ(x-μ)²/(n-1))

If you were to look at 777 digits (within the first 1000000) from anywhere in the file (pick any offset) you will notice that the digits are still highly normalized, similar variance and similar standard deviation !  The spread in fact will be almost identical;  perhaps one day there will be a rigorous proof of this conjecture.

[MarkHaysHarris777]

I updated the pi_digit_dist.c code in post #5.

[tllim]

I updated the pi_digit_dist.c code in post #5.

Thanks :-)