How to do hardware decoding of video?
#1
So I have a Rock64 board and I want to leverage its hardware decoding and encoding abilities in an Emby server.  But for a start I'd be happy just to get any sort of hardware decoding working.  I tried out Armbian Stretch and wasn't able to get either of my HEVC test files to play smoothly out of the box.  (vainfo returned an error when trying to get a driver name, so evidently something needs to be installed that's not, but I'm not sure what...)  And there's very little instruction out there that I can find, so I'm not sure if this is an Armbian shortcoming or just a matter of me not knowing how to get all the pieces in place.

I'm interested in ayufan's images, but don't know if it's any easier to get hardware decoding working there than on Armbian...  Has anyone gotten hardware decoding working on these boards?  What OS works best and what needs to be installed/configured to get it working properly?
  Reply
#2
(08-08-2018, 04:30 AM)SuperSaiyanCaleb Wrote: So I have a Rock64 board and I want to leverage its hardware decoding and encoding abilities in an Emby server....
I found in Emby server some code with "ffmpeg" and "DRM" API so setup your Emby server to use ffmpeg and DRM as output.
There are only few hw-accelerated possibilities - low-level MPP+DRM, gstreamer or ffmpeg API based players (including web browsers) like mpv (also in Kodi/LibreELEC), rkplayer, rkffplayer ... or simple acceleration testing utility.
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#3
Doesn't Emby use its built-in ffmpeg by default?  How would I go about configuring those two things?  I don't see anything in any of Emby's settings that mentions ffmpeg or drm.  Armbian came with mpv installed, but it wasn't able to decode an HEVC test file I tried using hardware decoding.  The player spat out this error before falling back to software decoding:

Code:
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)

Which seems to suggest to me that there is something I need to install to allow hardware decoding to work...  Are there specific OS images that have all the needed components working out of the box, or is there a particular OS that at least can have the needed components installed/configured with some simple instruction?
  Reply
#4
So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?
  Reply
#5
(08-12-2018, 02:45 AM)SuperSaiyanCaleb Wrote: So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?
As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).
The right questions:
  • Is your video player aware of this ?
  • Are you ready to configure your video player (ffmpeg based) to use hardware accelerated decoding ?
    • ffmpeg must be compiled with "--enable-rkmpp" and "--enable-libdrm"
    • video player must request "h264_rkmpp", "hevc_rkmpp", "vp8_rkmpp" or "vp9_rkmpp" decoder from ffmpeg
    • video player must handle/display "AV_PIX_FMT_DRM_PRIME" (DRM DMA buffers) format to render output to DRM plane (best with DRM atomic)
  • Are you ready to enhance/reprogram/redesign your video player to use hardware accelerated decoding chain ?
  • Are you able to contact developers of your video player to make this setup for you (for fee, beer or free) ?
If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#6
(08-12-2018, 04:55 AM)mcerveny Wrote:
(08-12-2018, 02:45 AM)SuperSaiyanCaleb Wrote: So is there no fleshed out procedure on how to use hardware decoding on this board in Linux yet?
As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).
The right questions:
  • Is your video player aware of this ?
  • Are you ready to configure your video player (ffmpeg based) to use hardware accelerated decoding ?
    • ffmpeg must be compiled with "--enable-rkmpp" and "--enable-libdrm"
    • video player must request "h264_rkmpp", "hevc_rkmpp", "vp8_rkmpp" or "vp9_rkmpp" decoder from ffmpeg
    • video player must handle/display "AV_PIX_FMT_DRM_PRIME" (DRM DMA buffers) format to render output to DRM plane (best with DRM atomic)
  • Are you ready to enhance/reprogram/redesign your video player to use hardware accelerated decoding chain ?
  • Are you able to contact developers of your video player to make this setup for you (for fee, beer or free) ?
If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).

I do not pretend to use some special player, but in the assemblies from Ayufan can not normally use even MPV or SM Player, only through some cunning manipulation.

You can immediately make the person started the video and it is not played immediately in full screen, where it is not clear how to manage it, and about working with the playlist, I'm silent at all
  Reply
#7
(08-12-2018, 04:55 AM)mcerveny Wrote: As I wrote hardware acceleration is fully supported on Rock64 (or many other Rockchip devices with MPP libraries).

So I can just slap Armbian on a microSD along with an HEVC-encoded MKV file and it will play as soon as I double-click the file or command mpv to play it? That's a rhetorical question, as I already know the answer is no. That's not a complaint, but apparently our definitions of "fully supported" differ. Since your post here was the first mention I've ever seen of MPP, I did some Googling on that and figured out how to hardware decode an HEVC file using ffmpeg and a null video output! (Currently am on ayufan's LXDE image.) But the same trick would not work with ffplay, and MPV would not play the file even if I specified the mpp decoder on the command line.

So tonight if I have time maybe I will see about switching back over to Armbian and see if I have any more luck with it. Some specific instructions on how to play a file with a given OS image would be awesome though...

Quote:If not, you should use MPV/KODI/rkplayer instead of your "randomly" chosen video player (from hundreds of available video players).

As I said, I'd be delighted to get videos playing on any media player you suggest. Also as I said before, and as continues to be the case so far, MPV hasn't worked. Or maybe I don't know what command to issue... Or what to install... Or what OS image works best... (All those things I was asking about in the first post...) Once I have playback working using the 'fully supported' method, I can see about working out my specific use cases after that. I just want to know what the elusive 'fully supported' method is... Undecided
  Reply
#8
(08-14-2018, 07:23 PM)SuperSaiyanCaleb Wrote: ...I just want to know what the elusive 'fully supported' method is...  Undecided
So. The same questions for linux distribution:
  • Does your distribution have right kernel with MPP/DRM support (configured with CONFIG_ROCKCHIP_MPP_SERVICE, CONFIG_ROCKCHIP_MPP_DEVICE, CONFIG_DRM_ROCKCHIP ...) ?
  • Does your distribution have right libraries with compiled support for hardware acceleration (ffmpeg (--enable-rkmpp, --enable-libdrm), libdrm-rockchip, mpp, gstreamer (ROCKCHIPMPP/VPUDEC/RKXIMAGE)) ?
  • Does your distribution have right version of mpv player with compiled MPP/DRM support (mpv (with options --hwdec=rkmpp/--vo=gpu|drm/--gpu-context=drm)) ?
If not, you should not use "randomly" chosen distribution with generic compiled packages.
Try Ayufan combination: https://github.com/ayufan-rock64/linux-build/releases (https://github.com/ayufan-rock64/linux-kernel/releases) + https://launchpad.net/~ayufan/+archive/u.../+packages (read also videoplayback notes)
Try LibreELEC for Rock64: https://test.libreelec.tv/ or http://kwiboo.libreelec.tv/test/
(if does not work buy beer to ayufan, Kwiboo and LongChair and ask for repair)
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply
#9
So finally had a chance to do some experimenting on this this weekend.  Followed the instructions in videoplayback notes when using ayufan's LXDE Bionic build, and when I attempt to play a file using rkmpv, I get a black screen, and this on the console:

Code:
Playing: test.mkv
(+) Video --vid=1 (*) (hevc 1920x1080 23.976fps)
(+) Audio --aid=1 --alang=eng (*) 'Surround' (aac 6ch 48000Hz)
    Audio --aid=2 --alang=eng 'Stereo' (aac 2ch 48000Hz)
    Audio --aid=3 --alang=fre 'Stereo' (aac 2ch 48000Hz)
    Audio --aid=4 --alang=spa 'Stereo' (aac 2ch 48000Hz)
    Subs  --sid=1 --slang=eng (hdmv_pgs_subtitle)
    Subs  --sid=2 --slang=fre (hdmv_pgs_subtitle)
    Subs  --sid=3 --slang=spa (hdmv_pgs_subtitle)
    Subs  --sid=4 --slang=spa (hdmv_pgs_subtitle)
    Subs  --sid=5 --slang=por (hdmv_pgs_subtitle)
    Subs  --sid=6 --slang=pol (hdmv_pgs_subtitle)
    Subs  --sid=7 --slang=dut (hdmv_pgs_subtitle)
    Subs  --sid=8 --slang=dan (hdmv_pgs_subtitle)
    Subs  --sid=9 --slang=fin (hdmv_pgs_subtitle)
    Subs  --sid=10 --slang=nor (hdmv_pgs_subtitle)
    Subs  --sid=11 --slang=swe (hdmv_pgs_subtitle)
(+) Subs  --sid=12 --slang=eng (*) (hdmv_pgs_subtitle)
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu/opengl] Could not choose EGLConfig!
mpi: mpp version: Without VCS, under bleeding
AO: [pulse] 48000Hz 5.1 6ch float
H265D_PARSER: No start code is found.
Using hardware decoding (rkmpp).
VO: [gpu] 1920x1080 drm_prime
[vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000
[vo/gpu/drmprime-drm] Failed to create framebuffer on layer 0.
AV: 00:00:00 / 01:01:37 (0%) A-V:  0.000

The failed framebuffer errors continue to print, with time indices gradually increasing, so the file appears to be "playing", only there is no video output.
  Reply
#10
???
  • downloaded and flashed bionic-lxde-rock64-0.7.9-1067-arm64.img
  • downloaded free movie bbb-1920x1080-cfg06.mkv
  • test run
    Code:
    $ ### alsa/plughw:CARD=I2S,DEV=0 audio output is analog jack
    $ ### only 3 frames dropped (10 min video played), no errors, smooth video play
    $ rkmpv --audio-device=alsa/plughw:CARD=I2S,DEV=0 bbb-1920x1080-cfg06.mkv
    Playing: bbb-1920x1080-cfg06.mkv
    (+) Video --vid=1 (*) (hevc 1920x1080 60.000fps)
    (+) Audio --aid=1 (*) (aac 6ch 48000Hz)
    [vo/gpu] Can't handle VT release - signal already used
    [vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
    [vo/gpu/opengl] Could not choose EGLConfig!
    mpi: mpp version: Without VCS, under bleeding
    AO: [alsa] 48000Hz stereo 2ch float
    H265D_PARSER: No start code is found.
    Using hardware decoding (rkmpp).
    VO: [gpu] 1920x1080 drm_prime[nv12]
    [vo/gpu] Using HW-overlay mode. No GL filtering is performed on the video!
    AV: 00:10:34 / 00:10:34 (99%) A-V:  0.000 Dropped: 3
  • verify video chain during video play
    Code:
    $ sudo bash

    # ### verify rkvdec is active (eg. HW decode)
    # cat /sys/kernel/debug/pm_genpd/pm_genpd_summary
    domain                          status          slaves
        /device                                             runtime status
    ----------------------------------------------------------------------
    pd_vpu                          off            
        /devices/platform/ff350800.iommu                    suspended
        /devices/platform/vpu_combo                         suspended
    pd_video                        on              
        /devices/platform/ff360480.iommu                    active
        /devices/platform/ff360000.rkvdec                   active
    pd_hevc                         on              
        /devices/platform/ff330200.iommu                    suspended
        /devices/platform/ff340800.iommu                    suspended
        /devices/platform/ff330000.h265e                    unsupported
        /devices/platform/ff340000.vepu                     unsupported

    # ### default clock running without overclocking (not needed for 1080p)
    # egrep 'armclk |aclk_rkvdec |cabac |vdec_core |ddrc ' /sys/kernel/debug/clk/clk_summary
                 armclk                       0            0   408000000          0 0  
                 aclk_rkvdec                  3            4   491519999          0 0  
              sclk_vdec_core                  2            3   300000000          0 0  
              sclk_vdec_cabac                 2            3   400000000          0 0  
              sclk_ddrc                       2            2   786000000          0 0  

    # ### video is rendered to "win1-0" (on foreground, hw scaled from 1080p to monitor resolution), "win0-0" is Xorg with LXDE (on background)
    # cat /sys/kernel/debug/dri/0/summary
    VOP [ff370000.vop]: ACTIVE
        Connector: HDMI-A
        overlay_mode[0] bus_format[100a] output_mode[f] color_space[0]
        Display mode: 1280x1024p60
        clk[108000] real_clk[108000] type[48] flag[5]
        H: 1280 1328 1440 1688
        V: 1024 1025 1028 1066
        win0-0: ACTIVE
        format: AR24 little-endian (0x34325241) SDR[0] color_space[0]
        csc: y2r[0] r2r[0] r2y[0] csc mode[0]
        zpos: 1
        src: pos[0x0] rect[1280x1024]
        dst: pos[0x0] rect[1280x1024]
        buf[0]: addr: 0x000000000299b000 pitch: 5120 offset: 0
        win1-0: ACTIVE
        format: NV12 little-endian (0x3231564e) SDR[0] color_space[3]
        csc: y2r[1] r2r[0] r2y[0] csc mode[1]
        zpos: 0
        src: pos[0x0] rect[1920x1080]
        dst: pos[0x152] rect[1280x720]
        buf[0]: addr: 0x0000000003b6d000 pitch: 2304 offset: 0
        buf[1]: addr: 0x0000000003b6d000 pitch: 2304 offset: 2488320
        win2-0: DISABLED
        post: sdr2hdr[0] hdr2sdr[0]
        pre : sdr2hdr[0]
        post CSC: r2y[0] y2r[0] CSC mode[1]
I left this community in Aug 2019 due to PINE64 refusal to produce/deliver ROCK64-1G version 3 after more than one year of changing statuses to "planning", "evaluating", "releasing", "availability", "estimated availability" and finally "no schedule" Angry. ROCK64 is dead platform without any advantage. Buy Raspberry PI 4 !
Away
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to Change Video Memory Amount gazhow64 0 685 12-10-2022, 04:12 PM
Last Post: gazhow64
  The state of mainline hardware decoding CounterPillow 17 14,664 01-26-2022, 03:39 PM
Last Post: sigmaris
  Hardware acceleration using FFmpeg gusarg81 0 2,817 08-21-2020, 01:36 PM
Last Post: gusarg81
  Hardware Status Monitoring? Leapo 7 14,892 03-15-2020, 08:44 AM
Last Post: bendem
Information New Builds from ayufan (0.8.X) w Video/3D/Mate desktop Luke 37 52,248 09-09-2019, 04:01 AM
Last Post: ayufan
  Video : Media Script for the Rock64/4K video in Linux NicoD 1 4,148 03-30-2019, 02:17 AM
Last Post: zet_lab
  hardware-accelerated video transcoding (Plex) on Rock64 mdr 2 7,219 02-07-2019, 03:42 PM
Last Post: mdr
  Running debian testing, web video blows :( dougunder 2 3,580 04-22-2018, 03:24 PM
Last Post: dougunder
  Available video players with hw decoder support dlu 27 42,121 03-01-2018, 11:45 AM
Last Post: elik745i
Lightbulb MPP+DRM: low level API for video codecs and display (also 4k UHD+HDR+HLG) mcerveny 0 2,976 01-27-2018, 11:23 AM
Last Post: mcerveny

Forum Jump:


Users browsing this thread: 1 Guest(s)