mainline kernel hardware acceleration
#21
linuxtv wip against media_tree master from linuxtv.org
drm next against next20200501 from kernel.org
ffmpeg v4l2 rkvdec against official 4.2.2

i made patchsets against next20200529 preparing for 5.7 to be released. (and prob slight changes afterwards). i was to share them when i get to my pbp (on phone now).

you are right that the patches didn't apply cleanly. i made several changes.

the kernel i am testing now, working well is:

next20200529
pinebookpro patch: https://pastebin.com/YK4LXeyv
LibreElec 1000 patch
v4l2 patch: https://pastebin.com/gaQSZQ6B
drm patch: https://pastebin.com/b1BWZ8Cg

plus:
ffmpeg 4.2.3 with v4l2-request-hwaccel-4.2.2-rkvdec patch: https://pastebin.com/uF5kpABD
kodi-git
libva-v4l2-request_master + pulls 29+30
libva_master + pulls 332+340
(for libva 332 delete the 5th chunk, version bump, or apply against libva 2.6.0)

general responsiveness is actually significantly better than 5.7-rc7 (!?), and my nvme drive is also much better handled. interesting.

also, i would recommend compiling hantro and rkvdec into the kernel, not modules, so that they are detected in the same order consistently.
#22
Thank You again for Your help @xmixahlx ! Hardware decoding is now working nicely, following the steps You outlined above. Using ffmpeg and null output:

Code:
> ffmpeg -hwaccel drm -i bbb_sunflower_1080p_60fps_normal.mp4 -f null -

ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Debian 9.3.0-13)
  configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request --enable-libdrm --enable-libudev --enable-openssl
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    creation_time   : 2013-12-16T17:59:32.000000Z
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    composer        : Sacha Goedegebure
  Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn, 120 tbc (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
    Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
    Side data:
      audio service type: main
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
  Stream #0:2 -> #0:1 (ac3 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: missing required mem2mem capability
    Last message repeated 1 times
[h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed
[h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed
frame=   10 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A    
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    composer        : Sacha Goedegebure
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    encoder         : Lavf58.29.100
    Stream #0:0(und): Video: wrapped_avframe, nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
      encoder         : Lavc58.54.100 wrapped_avframe
    Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1(side), s16, 4608 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
      encoder         : Lavc58.54.100 pcm_s16le
    Side data:
      audio service type: main
frame=   52 fps= 52 q=-0.0 size=N/A time=00:00:00.90 bitrate=N/A speed=0.899x    
frame=   97 fps= 64 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A speed=1.09x    
frame=  141 fps= 70 q=-0.0 size=N/A time=00:00:02.40 bitrate=N/A speed=1.19x    
frame=  187 fps= 74 q=-0.0 size=N/A time=00:00:03.15 bitrate=N/A speed=1.24x    
frame=  233 fps= 77 q=-0.0 size=N/A time=00:00:03.91 bitrate=N/A speed=1.29x    
frame=  283 fps= 80 q=-0.0 size=N/A time=00:00:04.75 bitrate=N/A speed=1.34x    
frame=  331 fps= 82 q=-0.0 size=N/A time=00:00:05.55 bitrate=N/A speed=1.37x    
frame=  378 fps= 83 q=-0.0 size=N/A time=00:00:06.33 bitrate=N/A speed=1.39x    
frame=  406 fps= 83 q=-0.0 Lsize=N/A time=00:00:06.80 bitrate=N/A speed= 1.4x    
video:213kB audio:3780kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

So I am already half way there. Now I only need to get video rendering working properly, because the output with mpv is still dropping frames. I did rebuild against patched ffmpeg, but running mpv with drm option for decoding and video output gives me errors and mpv does not play the video:


Code:
$> ~/mpv-build/mpv/build/mpv --hwdec=drm --vo=gpu --gpu-context=drm bbb_sunflower_1080p_60fps_normal.mp4

(+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
(+) Audio --aid=1 (*) (mp3 2ch 48000Hz)
    Audio --aid=2 (*) (ac3 6ch 48000Hz)
File tags:
Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
Composer: Sacha Goedegebure
Genre: Animation
Title: Big Buck Bunny, Sunflower version
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu] Failed to commit ModeSetting atomic request (-13)
[vo/gpu/opengl] Failed to set CRTC for connector 49: Permission denied
[vo/gpu] Failed initializing any suitable GPU context!
Error opening/initializing the selected video_out (--vo) device.
Video: no video

Exiting... (Errors when loading file)

Leaving off the "--gpu-context=drm" option will allow the file to play, but as I said output is dropping frames.
Do You happen to know what the issue here is? From Your posts I gather, You do not use mpv Yourself, but maybe You have come across something... In any case thank You very much again, this is already way better, than what I had working before!
#23
i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)

for quick test monitor top and use: "ffmpeg -re --hwaccel drm -i inputfile -f null -"

for hwaccel playback use kodi-git. see my other thread. i am also using jellyfin-kodi. works great playing files from my htpc/server.

5.7 is released so i will update patches soon.
#24
(06-01-2020, 09:19 AM)xmixahlx Wrote: i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)
...

Code:
$> ffmpeg -decoders | grep v4l
ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers
 built with gcc 9 (Debian 9.3.0-13)
 configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request
--enable-libdrm --enable-libudev --enable-openssl
 libavutil      56. 31.100 / 56. 31.100
 libavcodec     58. 54.100 / 58. 54.100
 libavformat    58. 29.100 / 58. 29.100
 libavdevice    58.  8.100 / 58.  8.100
 libavfilter     7. 57.100 /  7. 57.100
 libswscale      5.  5.100 /  5.  5.100
 libswresample   3.  5.100 /  3.  5.100
V..... h263_v4l2m2m         V4L2 mem2mem H.263 decoder wrapper (codec h263)
V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264)
V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc)
V..... mpeg1_v4l2m2m        V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video)
V..... mpeg2_v4l2m2m        V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video)
V..... mpeg4_v4l2m2m        V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4)
V..... vc1_v4l2m2m          V4L2 mem2mem VC1 decoder wrapper (codec vc1)
V..... vp8_v4l2m2m          V4L2 mem2mem VP8 decoder wrapper (codec vp8)
V..... vp9_v4l2m2m          V4L2 mem2mem VP9 decoder wrapper (codec vp9)

Edit: Isn't that the expected output? I see now all the *_v4l2request are missing. Sorry!

The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.
#25
Hi @xmixahlx

I would like to know where did you get these patches from ?
Is there any plan to upstream these?

I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?

I would love to test it too and build, package it for manjaro Smile

Keep up the good work.
Manjaro ARM Team.
Devices: Pinebook Pro & PinePhone.
#26
(06-01-2020, 09:24 AM)termac Wrote: The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.

for a better example of the kodi-gbm+drm+gles experience, use: "ffmpeg -re --hwaccel drm -hwaccel_output_format drm_prime -i inputfile -f null -"

you can also add pixel format with "-pix_fmt bgra" and output to framebuffer with "-f fbdev /dev/fb0"

(06-01-2020, 02:31 PM)spikerguy Wrote: Hi @xmixahlx

I would like to know where did you get these patches from ?
Is there any plan to upstream these?

I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?

I would love to test it too and build, package it for manjaro Smile

Keep up the good work.

@spikerguy @tsys

the drm is available in 5.7, but it is patched for upstream + WIP features. the v4l2 api is in constant flux. kwiboo upstreams some patches, and what's still WIP is included in libreelec. their build system pulls in patches and builds packages from source, which makes adapting their patches from their git repo easier than cherry picking patches from LKML (!!!).

in more detail: after a major release he updates his repo branches and gets ready to package for libreelec, then he makes a PR on the libreelec main git. when merged he deletes his branch.

you can generally make his patches yourself by diffing each branch to their base. (that's what i do between his major releases.)

a few additional WIP things are not included in the official patches. you can get those via the branch diffs.

https://patchwork.kernel.org/project/lin...ter=182393
https://github.com/Kwiboo/linux-rockchip
https://github.com/Kwiboo/LibreELEC.tv/t...ux/default

for packaging, you would also want to include ffmpeg-v4l2-request. (the diff against official 4.2.2 is above and applies to 4.2.3, also), and kodi-git built with gbm+gles (see my other thread).

ffmpeg-v4l2-request: https://github.com/Kwiboo/FFmpeg/tree/v4...ccel-4.2.2
ffmpeg-v4l2-request-rockchip: https://github.com/Kwiboo/FFmpeg/tree/v4...2.2-rkvdec
(have these conflict. rkvdec is experimental for rockchip, the other is stable for both rockchip and allwinner, but less featureful on rockchip.)


libva and libva-v4l2-request are somewhat broken at the moment and can be skipped for packaging right now.


edit: i attached my patches, config, and patchseries script (delete .log). you can see several of these made it into libreelec 5.7 branch with similar naming.


Attached Files
.log   5.7.0-kwiboo-patches.tar.lrz.log (Size: 218.25 KB / Downloads: 189)
#27
Thanks for the patches and kernel config @xmixahlx !

Trying to build gives me an error regarding firmware:

Code:
make KERNELRELEASE=5.7.0-pbp-custom ARCH=arm64 KBUILD_BUILD_VERSION=1 -f ./Makefile
 CALL    scripts/checksyscalls.sh
 CALL    scripts/atomic/check-atomics.sh
 CHK     include/generated/compile.h
make[7]: *** No rule to make target '/lib/firmware/rockchip/dptx.bin', needed by 'drivers/base/firmware_loader/builtin/rockchip/dptx.bin.gen.o'.  Stop.
make[6]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader/builtin] Error 2
make[5]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader] Error 2
make[4]: *** [scripts/Makefile.build:488: drivers/base] Error 2
make[3]: *** [Makefile:1729: drivers] Error 2
make[2]: *** [debian/rules:6: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 2
make: *** [Makefile:1460: bindeb-pkg] Error 2

Anyone got an idea if the file '/lib/firmware/rockchip/dptx.bin' is really needed (it is not present in my earlier kernel build trees) or what to do here?
#28
i build this into the kernel, but you can remove by deleting that line in the config. it is installed manually by danielt's installer, and included in debian's firmware-misc-nonfree package.
#29
Installing firmware-misc-nonfree did the trick. Thanks again!
#30
confirmed that the youtube kodi plugin working well with hwaccel (tested up to 1080p). h264 works great. vp9 has occasional artifacts. recommend using Adaptive H264 in MPEG DASH settings.

kodi-git thread updated and added inputstream.adaptive-git script for use with youtube-kodi-addon, etc.


Possibly Related Threads…
Thread Author Replies Views Last Post
  Kernel panics solostian 5 159 09-20-2021, 03:17 PM
Last Post: jiyong
  red/green power led not working with kernel 5.14 alchemist 3 100 09-20-2021, 01:01 AM
Last Post: alchemist
  Very slow glamor 2D acceleration hjalfi 3 743 09-06-2021, 12:54 PM
Last Post: jbrock
  Sound stuttering on Manjaro gnome after latest kernel upgrade? pjsf 0 295 07-13-2021, 10:37 PM
Last Post: pjsf
  Kernel panic on boot with danielt's Debian installer hjalfi 11 1,837 07-09-2021, 08:52 AM
Last Post: moonwalkers
  Kernel crash on Manjaro ARM mfashby 0 256 07-06-2021, 03:17 PM
Last Post: mfashby
  A true mainline Linux Kernel for the Pinebook Pro tsys 154 103,761 06-20-2021, 09:26 AM
Last Post: linuxad
  [LibreOffice & Gimp] ARM support: no hardware acceleration on [Manjaro] Pinebook Pro regivanx 0 426 06-15-2021, 04:04 AM
Last Post: regivanx
Question Building custom kernel (5.11.x) for Manjaro? ppafin 4 1,219 06-02-2021, 04:17 AM
Last Post: ppafin
Lightbulb Hardware video acceleration in Chromium on Manjaro siemsenit 3 877 06-01-2021, 09:51 AM
Last Post: siemsenit

Forum Jump:


Users browsing this thread: 1 Guest(s)