PINE64
How to do hardware decoding of video? - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: ROCK64 (https://forum.pine64.org/forumdisplay.php?fid=85)
+--- Forum: Linux on Rock64 (https://forum.pine64.org/forumdisplay.php?fid=88)
+--- Thread: How to do hardware decoding of video? (/showthread.php?tid=6373)

How to do hardware decoding of video? - SuperSaiyanCaleb - 08-08-2018

So I have a Rock64 board and I want to leverage its hardware decoding and encoding abilities in an Emby server.  But for a start I'd be happy just to get any sort of hardware decoding working.  I tried out Armbian Stretch and wasn't able to get either of my HEVC test files to play smoothly out of the box.  (vainfo returned an error when trying to get a driver name, so evidently something needs to be installed that's not, but I'm not sure what...)  And there's very little instruction out there that I can find, so I'm not sure if this is an Armbian shortcoming or just a matter of me not knowing how to get all the pieces in place.

I'm interested in ayufan's images, but don't know if it's any easier to get hardware decoding working there than on Armbian...  Has anyone gotten hardware decoding working on these boards?  What OS works best and what needs to be installed/configured to get it working properly?

RE: How to do hardware decoding of video? - mcerveny - 08-08-2018

(08-08-2018, 04:30 AM)SuperSaiyanCaleb Wrote: So I have a Rock64 board and I want to leverage its hardware decoding and encoding abilities in an Emby server....
I found in Emby server some code with "ffmpeg" and "DRM" API so setup your Emby server to use ffmpeg and DRM as output.
There are only few hw-accelerated possibilities - low-level MPP+DRM, gstreamer or ffmpeg API based players (including web browsers) like mpv (also in Kodi/LibreELEC), rkplayer, rkffplayer ... or simple acceleration testing utility.

RE: How to do hardware decoding of video? - SuperSaiyanCaleb - 08-09-2018

Doesn't Emby use its built-in ffmpeg by default?  How would I go about configuring those two things?  I don't see anything in any of Emby's settings that mentions ffmpeg or drm.  Armbian came with mpv installed, but it wasn't able to decode an HEVC test file I tried using hardware decoding.  The player spat out this error before falling back to software decoding:

Code:
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)

Which seems to suggest to me that there is something I need to install to allow hardware decoding to work...  Are there specific OS images that have all the needed components working out of the box, or is there a particular OS that at least can have the needed components installed/configured with some simple instruction?

RE: How to do hardware decoding of video? - SuperSaiyanCaleb - 08-12-2018

So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?

RE: How to do hardware decoding of video? - mcerveny - 08-12-2018

(08-12-2018, 02:45 AM)SuperSaiyanCaleb Wrote: So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?
As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).
The right questions:
  • Is your video player aware of this ?
  • Are you ready to configure your video player (ffmpeg based) to use hardware accelerated decoding ?
    • ffmpeg must be compiled with "--enable-rkmpp" and "--enable-libdrm"
    • video player must request "h264_rkmpp", "hevc_rkmpp", "vp8_rkmpp" or "vp9_rkmpp" decoder from ffmpeg
    • video player must handle/display "AV_PIX_FMT_DRM_PRIME" (DRM DMA buffers) format to render output to DRM plane (best with DRM atomic)
  • Are you ready to enhance/reprogram/redesign your video player to use hardware accelerated decoding chain ?
  • Are you able to contact developers of your video player to make this setup for you (for fee, beer or free) ?
If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).

RE: How to do hardware decoding of video? - zet_lab - 08-12-2018

(08-12-2018, 04:55 AM)mcerveny Wrote:
(08-12-2018, 02:45 AM)SuperSaiyanCaleb Wrote: So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?
As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).
The right questions:
  • Is your video player aware of this ?
  • Are you ready to configure your video player (ffmpeg based) to use hardware accelerated decoding ?
    • ffmpeg must be compiled with "--enable-rkmpp" and "--enable-libdrm"
    • video player must request "h264_rkmpp", "hevc_rkmpp", "vp8_rkmpp" or "vp9_rkmpp" decoder from ffmpeg
    • video player must handle/display "AV_PIX_FMT_DRM_PRIME" (DRM DMA buffers) format to render output to DRM plane (best with DRM atomic)
  • Are you ready to enhance/reprogram/redesign your video player to use hardware accelerated decoding chain ?
  • Are you able to contact developers of your video player to make this setup for you (for fee, beer or free) ?
If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).

I do not pretend to use some special player, but in the assemblies from Ayufan can not normally use even MPV or SM Player, only through some cunning manipulation.

You can immediately make the person started the video and it is not played immediately in full screen, where it is not clear how to manage it, and about working with the playlist, I'm silent at all

RE: How to do hardware decoding of video? - SuperSaiyanCaleb - 08-14-2018

(08-12-2018, 04:55 AM)mcerveny Wrote: As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).

So I can just slap Armbian on a microSD along with an HEVC-encoded MKV file and it will play as soon as I double-click the file or command mpv to play it? That's a rhetorical question, as I already know the answer is no. That's not a complaint, but apparently our definitions of "fully supported" differ. Since your post here was the first mention I've ever seen of MPP, I did some Googling on that and figured out how to hardware decode an HEVC file using ffmpeg and a null video output! (Currently am on ayufan's LXDE image.) But the same trick would not work with ffplay, and MPV would not play the file even if I specified the mpp decoder on the command line.

So tonight if I have time maybe I will see about switching back over to Armbian and see if I have any more luck with it. Some specific instructions on how to play a file with a given OS image would be awesome though...

Quote:If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).

As I said, I'd be delighted to get videos playing on any media player you suggest. Also as I said before, and as continues to be the case so far, MPV hasn't worked. Or maybe I don't know what command to issue... Or what to install... Or what OS image works best... (All those things I was asking about in the first post...) Once I have playback working using the 'fully supported' method, I can see about working out my specific use cases after that. I just want to know what the elusive 'fully supported' method is... Undecided

RE: How to do hardware decoding of video? - mcerveny - 08-15-2018

(08-14-2018, 07:23 PM)SuperSaiyanCaleb Wrote: ...I just want to know what the elusive 'fully supported' method is...  Undecided
So. The same questions for linux distribution:
  • Does your distribution have right kernel with MPP/DRM support (configured with CONFIG_ROCKCHIP_MPP_SERVICE, CONFIG_ROCKCHIP_MPP_DEVICE, CONFIG_DRM_ROCKCHIP ...) ?
  • Does your distribution have right libraries with compiled support for hardware acceleration (ffmpeg (--enable-rkmpp, --enable-libdrm), libdrm-rockchip, mpp, gstreamer (ROCKCHIPMPP/VPUDEC/RKXIMAGE)) ?
  • Does your distribution have right version of mpv player with compiled MPP/DRM support (mpv (with options --hwdec=rkmpp/--vo=gpu|drm/--gpu-context=drm)) ?
If not, you should not use "randomly" chosen distribution with generic compiled packages.
Try Ayufan combination: https://github.com/ayufan-rock64/linux-build/releases (https://github.com/ayufan-rock64/linux-kernel/releases) + https://launchpad.net/~ayufan/+archive/ubuntu/rock64-ppa/+packages (read also videoplayback notes)
Try LibreELEC for Rock64: https://test.libreelec.tv/ or http://kwiboo.libreelec.tv/test/
(if does not work buy beer to ayufan, Kwiboo and LongChair and ask for repair)

RE: How to do hardware decoding of video? - SuperSaiyanCaleb - 08-26-2018

So finally had a chance to do some experimenting on this this weekend.  Followed the instructions in videoplayback notes when using ayufan's LXDE Bionic build, and when I attempt to play a file using rkmpv, I get a black screen, and this on the console:

Code:
Playing: test.mkv
(+) Video --vid=1 (*) (hevc 1920x1080 23.976fps)
(+) Audio --aid=1 --alang=eng (*) 'Surround' (aac 6ch 48000Hz)
    Audio --aid=2 --alang=eng 'Stereo' (aac 2ch 48000Hz)
    Audio --aid=3 --alang=fre 'Stereo' (aac 2ch 48000Hz)
    Audio --aid=4 --alang=spa 'Stereo' (aac 2ch 48000Hz)
    Subs  --sid=1 --slang=eng (hdmv_pgs_subtitle)
    Subs  --sid=2 --slang=fre (hdmv_pgs_subtitle)
    Subs  --sid=3 --slang=spa (hdmv_pgs_subtitle)
    Subs  --sid=4 --slang=spa (hdmv_pgs_subtitle)
    Subs  --sid=5 --slang=por (hdmv_pgs_subtitle)
    Subs  --sid=6 --slang=pol (hdmv_pgs_subtitle)
    Subs  --sid=7 --slang=dut (hdmv_pgs_subtitle)
    Subs  --sid=8 --slang=dan (hdmv_pgs_subtitle)
    Subs  --sid=9 --slang=fin (hdmv_pgs_subtitle)
    Subs  --sid=10 --slang=nor (hdmv_pgs_subtitle)
    Subs  --sid=11 --slang=swe (hdmv_pgs_subtitle)
(+) Subs  --sid=12 --slang=eng (*) (hdmv_pgs_subtitle)
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl] Could not choose EGLConfig!
mpi: mpp version: Without VCS, under bleeding
AO: [pulse] 48000Hz 5.1 6ch float
H265D_PARSER: No start code is found.
Using hardware decoding (rkmpp).
VO: [gpu] 1920x1080 drm_prime
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000

The failed framebuffer errors continue to print, with time indices gradually increasing, so the file appears to be "playing", only there is no video output.

RE: How to do hardware decoding of video? - mcerveny - 08-28-2018

???
  • downloaded and flashed bionic-lxde-rock64-0.7.9-1067-arm64.img
  • downloaded free movie bbb-1920x1080-cfg06.mkv
  • test run
    Code:
    $ ### alsa/plughw:CARD=I2S,DEV=0 audio output is analog jack
    $ ### only 3 frames dropped (10 min video played), no errors, smooth video play
    $ rkmpv --audio-device=alsa/plughw:CARD=I2S,DEV=0 bbb-1920x1080-cfg06.mkv
    Playing: bbb-1920x1080-cfg06.mkv
    (+) Video --vid=1 (*) (hevc 1920x1080 60.000fps)
    (+) Audio --aid=1 (*) (aac 6ch 48000Hz)
    [vo/gpu] Can't handle VT release - signal already used
    [vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
    [vo/gpu/opengl] Could not choose EGLConfig!
    mpi: mpp version: Without VCS, under bleeding
    AO: [alsa] 48000Hz stereo 2ch float
    H265D_PARSER: No start code is found.
    Using hardware decoding (rkmpp).
    VO: [gpu] 1920x1080 drm_prime[nv12]
    [vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
    AV: 00:10:34 / 00:10:34 (99%) A-V:  0.000 Dropped: 3
  • verify video chain during video play
    Code:
    $ sudo bash

    # ### verify rkvdec is active (eg. HW decode)
    # cat /sys/kernel/debug/pm_genpd/pm_genpd_summary
    domain                          status          slaves
        /device                                             runtime status
    ----------------------------------------------------------------------
    pd_vpu                          off            
        /devices/platform/ff350800.iommu                    suspended
        /devices/platform/vpu_combo                         suspended
    pd_video                        on              
        /devices/platform/ff360480.iommu                    active
        /devices/platform/ff360000.rkvdec                   active
    pd_hevc                         on              
        /devices/platform/ff330200.iommu                    suspended
        /devices/platform/ff340800.iommu                    suspended
        /devices/platform/ff330000.h265e                    unsupported
        /devices/platform/ff340000.vepu                     unsupported

    # ### default clock running without overclocking (not needed for 1080p)
    # egrep 'armclk |aclk_rkvdec |cabac |vdec_core |ddrc ' /sys/kernel/debug/clk/clk_summary
                 armclk                       0            0   408000000          0 0  
                 aclk_rkvdec                  3            4   491519999          0 0  
              sclk_vdec_core                  2            3   300000000          0 0  
              sclk_vdec_cabac                 2            3   400000000          0 0  
              sclk_ddrc                       2            2   786000000          0 0  

    # ### video is rendered to "win1-0" (on foreground, hw scaled from 1080p to monitor resolution), "win0-0" is Xorg with LXDE (on background)
    # cat /sys/kernel/debug/dri/0/summary
    VOP [ff370000.vop]: ACTIVE
        Connector: HDMI-A
        overlay_mode[0] bus_format[100a] output_mode[f] color_space[0]
        Display mode: 1280x1024p60
        clk[108000] real_clk[108000] type[48] flag[5]
        H: 1280 1328 1440 1688
        V: 1024 1025 1028 1066
        win0-0: ACTIVE
        format: AR24 little-endian (0x34325241) SDR[0] color_space[0]
        csc: y2r[0] r2r[0] r2y[0] csc mode[0]
        zpos: 1
        src: pos[0x0] rect[1280x1024]
        dst: pos[0x0] rect[1280x1024]
        buf[0]: addr: 0x000000000299b000 pitch: 5120 offset: 0
        win1-0: ACTIVE
        format: NV12 little-endian (0x3231564e) SDR[0] color_space[3]
        csc: y2r[1] r2r[0] r2y[0] csc mode[1]
        zpos: 0
        src: pos[0x0] rect[1920x1080]
        dst: pos[0x152] rect[1280x720]
        buf[0]: addr: 0x0000000003b6d000 pitch: 2304 offset: 0
        buf[1]: addr: 0x0000000003b6d000 pitch: 2304 offset: 2488320
        win2-0: DISABLED
        post: sdr2hdr[0] hdr2sdr[0]
        pre : sdr2hdr[0]
        post CSC: r2y[0] y2r[0] CSC mode[1]