HW image transitions in Python
#1
Working on Rock64/Debian Stretch and trying to crossfade two images. I've tried Pygame, Kivy, and just basic SDL, but it seems like none of these are able to do anything other than SW mode. Is there a way of making them use HW or some other method of displaying images through Python where I'd be able to get a smooth realtime transition?
  Reply
#2
It depends:
  • alfa channel is sufficient for composition, two frames with scaling and positioning, only on video output -> use DRM (libdrm ...)
  • more complex compositions -> use OpenGLES 2.0 with shader program (libGLESv2/libEGL ...) (Search for OpenGLES python bindings and examples for RasberryPI.)
You must use libraries with support for ROCK64 HW and compatible kernel drivers - eg. libdrm rockchip video output and/or OpenGLES for mali-450 (utgard) for example.
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#3
Bloody hell this looks like some useful stuff that I haven't seen before! Thank you, I'll try this out.
  Reply
#4
So I'm still having issues trying to get this set up. When I try building libdrm, it works but says it will not be compiled with Rockchip API. The OpenGLES for mali-450 seems to go on just fine. Is there something I'm supposed to do to make sure it becomes a default or anything like that? I've tested with Glmark2, which I configure for drm-glesv2. It still only shows Mali when it loads. I did find RPIGL, which looks like it will work for me once this is able to use the hardware properly.

EDIT: Just flashed an image back to before any of this so I can start clean and try again. Trying to configure glmark, the first part checks for several things, including gbm, egl, and glesv2. It will fail until I install the Rockchip/Mali stuff, but then when it tries to build glmark I end up getting errors that it cannot find lgbm, legl, and lglesv2. If I put on the mesa versions then it will find them, so it seems like maybe they are getting installed but just not being pointed to properly. Is there a specific way to point to them?
  Reply
#5
I'm not getting anywhere with this. The drivers don't seem to be used, I've found this version of xserver and this kernel, both of which are Rockchip specific and say they'll be useful in making HW acceleration work. I think they're both a couple of jerks. They fail to install, and neither of which are for lack of dependencies. Anyone have more insight? I'm no expert on getting these things set up, but this is pretty ridiculous and definitely falls well within the realm of things I think should have just been working when I bought the stupid board.
  Reply
#6
(11-21-2018, 07:05 PM)gaggleoxfoggy Wrote: I'm not getting anywhere with this...
Step by step:
  1. use supported image - https://github.com/ayufan-rock64/linux-build/releases/ bionic-lxde-rock64-0.7.9-1067-arm64.img.xz
  2. login to lxde as rock64/rock64 and check installed packages
    Code:
    $ sudo bash
    # apt list --installed | egrep 'utgard|rockchip'
    ...
    libdrm-rockchip1/bionic,now 2.4.89-1ayufan1 arm64 [installed]
    libmali-rk-utgard-450-r7p0/bionic,now 1.6-1ayufan10 arm64 [installed]
    librockchip-mpp1/bionic,now 1.4.0-1ayufan3-2 arm64 [installed,automatic]
    linux-headers-4.4.132-1075-rockchip-ayufan-ga83beded8524/unknown,now 4.4.132-1075-rockchip-ayufan arm64 [installed,automatic]
    linux-image-4.4.132-1075-rockchip-ayufan-ga83beded8524/unknown,now 4.4.132-1075-rockchip-ayufan arm64 [installed,automatic]
    ...
    # dmesg | egrep -i 'drm|mali|utgard'
    ...
    [    1.377772] [drm] Rockchip DRM driver version: v1.0.1
    [    2.874168] I : [File] : drivers/gpu/arm/mali400/mali/linux/mali_kernel_linux.c; [Line] : 417; [Func] : mali_module_init(); svn_rev_string_from_arm of this mali_ko is '-a83beded8524', rk_ko_ver is '5', built at '08:21:11', on 'Jul 26 2018'.
    ...
  3. install required development packages
    Code:
    # apt install python pkg-config libjpeg-dev libpng-dev xserver-xorg-dev/bionic libmali-rk-dev
    # ls -l /usr/lib/aarch64-linux-gnu/ | egrep 'libEGL|libGLESv2'
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libEGL.so -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libEGL.so.1 -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libEGL.so.1.0.0 -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libEGL.so.1.4 -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libGLESv2.so -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libGLESv2.so.2 -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libGLESv2.so.2.0 -> libMali.so
    lrwxrwxrwx  1 root root       10 Mar  4  2018 libGLESv2.so.2.0.0 -> libMali.so
    # ls -l /usr/include/EGL/ /usr/include/GLES2
    /usr/include/EGL/:
    total 80
    -rw-r--r-- 1 root root 15361 Nov 19  2017 egl.h
    -rw-r--r-- 1 root root 59487 Nov 19  2017 eglext.h
    -rw-r--r-- 1 root root  3839 Nov 19  2017 eglplatform.h
    /usr/include/GLES2:
    total 256
    -rw-r--r-- 1 root root  43972 Nov 19  2017 gl2.h
    -rw-r--r-- 1 root root 210702 Nov 19  2017 gl2ext.h
    -rw-r--r-- 1 root root   1160 Nov 19  2017 gl2platform.h
  4. download, configure, compile, install glmark2 from sources
    Code:
    # git clone --depth 1 https://github.com/glmark2/glmark2.git
    # cd glmark2
    # ./waf configure --with-flavors=x11-glesv2
    Setting top to                           : /home/rock64/glmark2
    Setting out to                           : /home/rock64/glmark2/build
    Checking for 'gcc' (C compiler)          : /usr/bin/gcc
    Checking for 'g++' (C++ compiler)        : /usr/bin/g++
    Checking for header stdlib.h             : yes
    Checking for header string.h             : yes
    Checking for header unistd.h             : yes
    Checking for header stdint.h             : yes
    Checking for header stdio.h              : yes
    Checking for header jpeglib.h            : yes
    Checking for library m                   : yes
    Checking for library jpeg                : yes
    Checking for function memset             : yes
    Checking for function sqrt               : yes
    Checking for program 'pkg-config'        : /usr/bin/pkg-config
    Checking for 'libpng12'                  : not found
    Checking for 'libpng15'                  : not found
    Checking for 'libpng16'                  : yes
    Checking for 'x11'                       : yes
    Checking for 'gl'                        : not found
    Checking for 'egl'                       : yes
    Checking for 'glesv2'                    : yes
    Checking for 'libdrm'                    : yes
    Checking for 'gbm'                       : yes
    Checking for 'libudev'                   : not found
    Checking for 'mirclient'                 : yes
    Checking for 'wayland-client'            : not found
    Checking for 'wayland-egl'               : yes
    Prefix                                   : /usr/local
    Data path                                : /usr/local/share/glmark2
    Including extras                         : No
    Building flavors                         : ['x11-glesv2']
    'configure' finished successfully (5.990s)
    # ./waf
    Waf: Entering directory `/home/rock64/glmark2/build'
    [ 1/67] Compiling src/main.cpp
    [ 2/67] Compiling src/canvas-generic.cpp
    [ 3/67] Compiling src/native-state-x11.cpp
    ...
    # ./waf install
  5. run "glmark2-es2" (performance results are dependend onscreen/offcreen, gpu/memory/cpu clock ...)
    Code:
    $ /usr/local/bin/glmark2-es2
    =======================================================
        glmark2 2017.07
    =======================================================
        OpenGL Information
        GL_VENDOR:     ARM
        GL_RENDERER:   Mali-450 MP
        GL_VERSION:    OpenGL ES 2.0
    =======================================================
    [build] use-vbo=false: FPS: 50 FrameTime: 20.000 ms
    [build] use-vbo=true: FPS: 50 FrameTime: 20.000 ms
    ...
    [loop] fragment-steps=5:fragment-uniform=true:vertex-steps=5: FPS: 40 FrameTime: 25.000 ms
    =======================================================
                                      glmark2 Score: 40
    =======================================================
    $ /usr/local/bin/glmark2-es2
    =======================================================
        glmark2 2017.07
    =======================================================
        OpenGL Information
        GL_VENDOR:     ARM
        GL_RENDERER:   Mali-450 MP
        GL_VERSION:    OpenGL ES 2.0
    =======================================================
    [build] use-vbo=false: FPS: 196 FrameTime: 5.102 ms
    [build] use-vbo=true: FPS: 224 FrameTime: 4.464 ms
    ...
    [loop] fragment-steps=5:fragment-uniform=true:vertex-steps=5: FPS: 175 FrameTime: 5.714 ms
    =======================================================
                                      glmark2 Score: 165
    =======================================================

Now OpenGLESv2 development platform is verified to be working.
And remember OpenGLES is not OpenGL !

Buy me a Beer.
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#7
(11-21-2018, 07:05 PM)gaggleoxfoggy Wrote: I'm not getting anywhere with this.

You should check MPP (HW accelerated video encoding/decoding) and direct DRM (output image composing). DRM windows for rk3328 are described here (eg. win0 - SDR/HDR (best for video output), win1 - SDR (best for video OSD) and win2 - SDR with limited size (best for cursor)). I prepared working proof of concept example (post). You can use bionic-lxde-rock64-0.7.9-1067-arm64.img.xz.
  1. check/prepare developent environment
    Code:
    # apt list --installed | egrep 'mpp|libdrm-rockchip'
    libdrm-rockchip1/bionic,now 2.4.89-1ayufan1 arm64 [installed]
    librockchip-mpp1/bionic,now 1.4.0-1ayufan3-2 arm64 [installed,automatic]
    # apt install librockchip-mpp-dev libdrm-dev
    # ls -l /usr/lib/aarch64-linux-gnu/libdrm.so /usr/lib/aarch64-linux-gnu/libdrm_rockchip.so /usr/lib/aarch64-linux-gnu/librockchip_mpp.so
    lrwxrwxrwx 1 root root 15 Dec 19  2017 /usr/lib/aarch64-linux-gnu/libdrm.so -> libdrm.so.2.4.0
    lrwxrwxrwx 1 root root 24 Dec 19  2017 /usr/lib/aarch64-linux-gnu/libdrm_rockchip.so -> libdrm_rockchip.so.1.0.0
    lrwxrwxrwx 1 root root 20 Mar  2  2018 /usr/lib/aarch64-linux-gnu/librockchip_mpp.so -> librockchip_mpp.so.1
    # ls /usr/include/rockchip/ /usr/include/libdrm/
    /usr/include/libdrm/:
    ... drm_mode.h ... rockchip_rga.h ... drm.h  ... rockchip_drm.h .... drm_fourcc.h  ... rockchip_drmif.h  ...
    /usr/include/rockchip/:
    mpp_buffer.h  mpp_err.h  mpp_frame.h  mpp_meta.h  mpp_packet.h  mpp_task.h  rk_mpi.h  rk_mpi_cmd.h  rk_type.h  vpu.h  vpu_api.h
  2. compile and check MPP/DRM example (beware - working directly to drm framebuffer (eg. working without xserver and replace original FB when running)).
    Code:
    # git clone https://github.com/mcerveny/utils.git
    # cd utils
    # gcc -o mpi_dec -I/usr/include/rockchip -I/usr/include/libdrm -g -Wno-multichar mpi_dec.c -lrockchip_mpp -lpthread -ldrm
    # ./mpi_dec 2k60.hevc 16777220

Now there are verified MPP and DRM HW acceleration libraries.
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#8
Thank you so much for your help on this!

I've been able to verify HW working in all examples you provided. I still don't quite understand how to turn this into something usable, as any examples that have used the HW were not in Python, which is the only language I know at all... still just learning that one. Using your example, I could potentially just process my image transitions into videos beforehand and play them back when needed. But so far I've failed to get this running with pogles, kivy, pyqt5, or rpigl. RPIGL causes a segmentation fault as soon as it tries to create the window, and I haven't been able to get pogles, pyqt5, or kivy to even install. They all fail, kivy can't find GL, pyqt5 (specifically installing qt open source version) I can't even make sense of why its failing, and pogles seems to have an issue with my sip. I may have configured sip for the wrong platform, but even still I'd have a very hard time trying to integrate pogles into my program. The RPIGL examples I can make sense of enough to make it do what I need, and I've done a test in kivy before that I could work with. But is this where I'm misunderstanding GL vs GLES? Kivy says it uses OpenGL ES, but still seems hung up on GL. PyQt5 seems to me the most promising if I can make it install, but are these all just lost causes trying to make run?

EDIT: And looking back through the QT make log it looks like it is getting hung up on GL stuff as well. Which comes back to, am I misunderstanding how these work or do they just need to be configured a different way?
  Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  Python GPIO Library for the Rock64 (R64.GPIO) Leapo 32 13,825 05-12-2019, 02:46 PM
Last Post: tllim
Brick How to Write an Image to the eMMC Module Tutorial MarkHaysHarris777 55 24,694 02-06-2019, 04:02 PM
Last Post: tllim
  Rock64 scipy, numpy, scikit-image, matplotlib issue with pip3 ictele 0 157 02-05-2019, 05:41 PM
Last Post: ictele
  flash image to sd/emmc through ethernet LMM 0 218 02-04-2019, 07:36 AM
Last Post: LMM
  New User - No image file in LibreELEC download Frogmella 9 850 06-11-2018, 01:01 PM
Last Post: Frogmella
  SDL2 and LXDE image Max11 0 360 05-24-2018, 01:51 AM
Last Post: Max11
  flash image directly to eMMC? gmruiz 4 1,995 01-09-2018, 05:43 AM
Last Post: Luke
  ioBroker IoT image Luke 0 600 11-02-2017, 03:46 AM
Last Post: Luke
Brick Rock64 Linux Image Writing Scripts Codes and Tutorial MarkHaysHarris777 4 3,757 09-09-2017, 04:03 PM
Last Post: freebsd
  Need help build Volumio Image for Rock64 with sound hat btwozaunt 7 1,397 09-07-2017, 12:27 PM
Last Post: tllim

Forum Jump:


Users browsing this thread: 1 Guest(s)