mainline kernel hardware acceleration
#21
linuxtv wip against media_tree master from linuxtv.org
drm next against next20200501 from kernel.org
ffmpeg v4l2 rkvdec against official 4.2.2

i made patchsets against next20200529 preparing for 5.7 to be released. (and prob slight changes afterwards). i was to share them when i get to my pbp (on phone now).

you are right that the patches didn't apply cleanly. i made several changes.

the kernel i am testing now, working well is:

next20200529
pinebookpro patch: https://pastebin.com/YK4LXeyv
LibreElec 1000 patch
v4l2 patch: https://pastebin.com/gaQSZQ6B
drm patch: https://pastebin.com/b1BWZ8Cg

plus:
ffmpeg 4.2.3 with v4l2-request-hwaccel-4.2.2-rkvdec patch: https://pastebin.com/uF5kpABD
kodi-git
libva-v4l2-request_master + pulls 29+30
libva_master + pulls 332+340
(for libva 332 delete the 5th chunk, version bump, or apply against libva 2.6.0)

general responsiveness is actually significantly better than 5.7-rc7 (!?), and my nvme drive is also much better handled. interesting.

also, i would recommend compiling hantro and rkvdec into the kernel, not modules, so that they are detected in the same order consistently.
#22
Thank You again for Your help @xmixahlx ! Hardware decoding is now working nicely, following the steps You outlined above. Using ffmpeg and null output:

Code:
> ffmpeg -hwaccel drm -i bbb_sunflower_1080p_60fps_normal.mp4 -f null - ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers   built with gcc 9 (Debian 9.3.0-13)   configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request --enable-libdrm --enable-libudev --enable-openssl   libavutil      56. 31.100 / 56. 31.100   libavcodec     58. 54.100 / 58. 54.100   libavformat    58. 29.100 / 58. 29.100   libavdevice    58.  8.100 / 58.  8.100   libavfilter     7. 57.100 /  7. 57.100   libswscale      5.  5.100 /  5.  5.100   libswresample   3.  5.100 /  3.  5.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':   Metadata:     major_brand     : isom     minor_version   : 1     compatible_brands: isomavc1     creation_time   : 2013-12-16T17:59:32.000000Z     title           : Big Buck Bunny, Sunflower version     artist          : Blender Foundation 2008, Janus Bager Kristensen 2013     comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net     genre           : Animation     composer        : Sacha Goedegebure   Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s     Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn, 120 tbc (default)     Metadata:       creation_time   : 2013-12-16T17:59:32.000000Z       handler_name    : GPAC ISO Video Handler     Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)     Metadata:       creation_time   : 2013-12-16T17:59:37.000000Z       handler_name    : GPAC ISO Audio Handler     Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)     Metadata:       creation_time   : 2013-12-16T17:59:37.000000Z       handler_name    : GPAC ISO Audio Handler     Side data:       audio service type: main Stream mapping:   Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))   Stream #0:2 -> #0:1 (ac3 (native) -> pcm_s16le (native)) Press [q] to stop, [?] for help [h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: missing required mem2mem capability     Last message repeated 1 times [h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10 [h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed [h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10 [h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed frame=   10 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A     Output #0, null, to 'pipe:':   Metadata:     major_brand     : isom     minor_version   : 1     compatible_brands: isomavc1     composer        : Sacha Goedegebure     title           : Big Buck Bunny, Sunflower version     artist          : Blender Foundation 2008, Janus Bager Kristensen 2013     comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net     genre           : Animation     encoder         : Lavf58.29.100     Stream #0:0(und): Video: wrapped_avframe, nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)     Metadata:       creation_time   : 2013-12-16T17:59:32.000000Z       handler_name    : GPAC ISO Video Handler       encoder         : Lavc58.54.100 wrapped_avframe     Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1(side), s16, 4608 kb/s (default)     Metadata:       creation_time   : 2013-12-16T17:59:37.000000Z       handler_name    : GPAC ISO Audio Handler       encoder         : Lavc58.54.100 pcm_s16le     Side data:       audio service type: main frame=   52 fps= 52 q=-0.0 size=N/A time=00:00:00.90 bitrate=N/A speed=0.899x     frame=   97 fps= 64 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A speed=1.09x     frame=  141 fps= 70 q=-0.0 size=N/A time=00:00:02.40 bitrate=N/A speed=1.19x     frame=  187 fps= 74 q=-0.0 size=N/A time=00:00:03.15 bitrate=N/A speed=1.24x     frame=  233 fps= 77 q=-0.0 size=N/A time=00:00:03.91 bitrate=N/A speed=1.29x     frame=  283 fps= 80 q=-0.0 size=N/A time=00:00:04.75 bitrate=N/A speed=1.34x     frame=  331 fps= 82 q=-0.0 size=N/A time=00:00:05.55 bitrate=N/A speed=1.37x     frame=  378 fps= 83 q=-0.0 size=N/A time=00:00:06.33 bitrate=N/A speed=1.39x     frame=  406 fps= 83 q=-0.0 Lsize=N/A time=00:00:06.80 bitrate=N/A speed= 1.4x     video:213kB audio:3780kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown Exiting normally, received signal 2.

So I am already half way there. Now I only need to get video rendering working properly, because the output with mpv is still dropping frames. I did rebuild against patched ffmpeg, but running mpv with drm option for decoding and video output gives me errors and mpv does not play the video:


Code:
$> ~/mpv-build/mpv/build/mpv --hwdec=drm --vo=gpu --gpu-context=drm bbb_sunflower_1080p_60fps_normal.mp4 (+) Video --vid=1 (*) (h264 1920x1080 60.000fps) (+) Audio --aid=1 (*) (mp3 2ch 48000Hz)     Audio --aid=2 (*) (ac3 6ch 48000Hz) File tags: Artist: Blender Foundation 2008, Janus Bager Kristensen 2013 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net Composer: Sacha Goedegebure Genre: Animation Title: Big Buck Bunny, Sunflower version [vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device [vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable. [vo/gpu] Failed to commit ModeSetting atomic request (-13) [vo/gpu/opengl] Failed to set CRTC for connector 49: Permission denied [vo/gpu] Failed initializing any suitable GPU context! Error opening/initializing the selected video_out (--vo) device. Video: no video Exiting... (Errors when loading file)

Leaving off the "--gpu-context=drm" option will allow the file to play, but as I said output is dropping frames.
Do You happen to know what the issue here is? From Your posts I gather, You do not use mpv Yourself, but maybe You have come across something... In any case thank You very much again, this is already way better, than what I had working before!
#23
i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)

for quick test monitor top and use: "ffmpeg -re --hwaccel drm -i inputfile -f null -"

for hwaccel playback use kodi-git. see my other thread. i am also using jellyfin-kodi. works great playing files from my htpc/server.

5.7 is released so i will update patches soon.
#24
(06-01-2020, 09:19 AM)xmixahlx Wrote: i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)
...

Code:
$> ffmpeg -decoders | grep v4l ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers  built with gcc 9 (Debian 9.3.0-13)  configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request --enable-libdrm --enable-libudev --enable-openssl  libavutil      56. 31.100 / 56. 31.100  libavcodec     58. 54.100 / 58. 54.100  libavformat    58. 29.100 / 58. 29.100  libavdevice    58.  8.100 / 58.  8.100  libavfilter     7. 57.100 /  7. 57.100  libswscale      5.  5.100 /  5.  5.100  libswresample   3.  5.100 /  3.  5.100 V..... h263_v4l2m2m         V4L2 mem2mem H.263 decoder wrapper (codec h263) V..... h264_v4l2m2m         V4L2 mem2mem H.264 decoder wrapper (codec h264) V..... hevc_v4l2m2m         V4L2 mem2mem HEVC decoder wrapper (codec hevc) V..... mpeg1_v4l2m2m        V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video) V..... mpeg2_v4l2m2m        V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video) V..... mpeg4_v4l2m2m        V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4) V..... vc1_v4l2m2m          V4L2 mem2mem VC1 decoder wrapper (codec vc1) V..... vp8_v4l2m2m          V4L2 mem2mem VP8 decoder wrapper (codec vp8) V..... vp9_v4l2m2m          V4L2 mem2mem VP9 decoder wrapper (codec vp9)

Edit: Isn't that the expected output? I see now all the *_v4l2request are missing. Sorry!

The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.
#25
Hi @xmixahlx

I would like to know where did you get these patches from ?
Is there any plan to upstream these?

I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?

I would love to test it too and build, package it for manjaro Smile

Keep up the good work.
Manjaro ARM Team.
Devices: Pinebook Pro & PinePhone.
#26
(06-01-2020, 09:24 AM)termac Wrote: The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.

for a better example of the kodi-gbm+drm+gles experience, use: "ffmpeg -re --hwaccel drm -hwaccel_output_format drm_prime -i inputfile -f null -"

you can also add pixel format with "-pix_fmt bgra" and output to framebuffer with "-f fbdev /dev/fb0"

(06-01-2020, 02:31 PM)spikerguy Wrote: Hi @xmixahlx

I would like to know where did you get these patches from ?
Is there any plan to upstream these?

I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?

I would love to test it too and build, package it for manjaro Smile

Keep up the good work.

@spikerguy @tsys

the drm is available in 5.7, but it is patched for upstream + WIP features. the v4l2 api is in constant flux. kwiboo upstreams some patches, and what's still WIP is included in libreelec. their build system pulls in patches and builds packages from source, which makes adapting their patches from their git repo easier than cherry picking patches from LKML (!!!).

in more detail: after a major release he updates his repo branches and gets ready to package for libreelec, then he makes a PR on the libreelec main git. when merged he deletes his branch.

you can generally make his patches yourself by diffing each branch to their base. (that's what i do between his major releases.)

a few additional WIP things are not included in the official patches. you can get those via the branch diffs.

https://patchwork.kernel.org/project/lin...ter=182393
https://github.com/Kwiboo/linux-rockchip
https://github.com/Kwiboo/LibreELEC.tv/t...ux/default

for packaging, you would also want to include ffmpeg-v4l2-request. (the diff against official 4.2.2 is above and applies to 4.2.3, also), and kodi-git built with gbm+gles (see my other thread).

ffmpeg-v4l2-request: https://github.com/Kwiboo/FFmpeg/tree/v4...ccel-4.2.2
ffmpeg-v4l2-request-rockchip: https://github.com/Kwiboo/FFmpeg/tree/v4...2.2-rkvdec
(have these conflict. rkvdec is experimental for rockchip, the other is stable for both rockchip and allwinner, but less featureful on rockchip.)


libva and libva-v4l2-request are somewhat broken at the moment and can be skipped for packaging right now.


edit: i attached my patches, config, and patchseries script (delete .log). you can see several of these made it into libreelec 5.7 branch with similar naming.


Attached Files
.log   5.7.0-kwiboo-patches.tar.lrz.log (Size: 218.25 KB / Downloads: 785)
#27
Thanks for the patches and kernel config @xmixahlx !

Trying to build gives me an error regarding firmware:

Code:
make KERNELRELEASE=5.7.0-pbp-custom ARCH=arm64 KBUILD_BUILD_VERSION=1 -f ./Makefile  CALL    scripts/checksyscalls.sh  CALL    scripts/atomic/check-atomics.sh  CHK     include/generated/compile.h make[7]: *** No rule to make target '/lib/firmware/rockchip/dptx.bin', needed by 'drivers/base/firmware_loader/builtin/rockchip/dptx.bin.gen.o'.  Stop. make[6]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader/builtin] Error 2 make[5]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader] Error 2 make[4]: *** [scripts/Makefile.build:488: drivers/base] Error 2 make[3]: *** [Makefile:1729: drivers] Error 2 make[2]: *** [debian/rules:6: build] Error 2 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 2 make: *** [Makefile:1460: bindeb-pkg] Error 2

Anyone got an idea if the file '/lib/firmware/rockchip/dptx.bin' is really needed (it is not present in my earlier kernel build trees) or what to do here?
#28
i build this into the kernel, but you can remove by deleting that line in the config. it is installed manually by danielt's installer, and included in debian's firmware-misc-nonfree package.
#29
Installing firmware-misc-nonfree did the trick. Thanks again!
#30
confirmed that the youtube kodi plugin working well with hwaccel (tested up to 1080p). h264 works great. vp9 has occasional artifacts. recommend using Adaptive H264 in MPEG DASH settings.

kodi-git thread updated and added inputstream.adaptive-git script for use with youtube-kodi-addon, etc.


Possibly Related Threads…
Thread Author Replies Views Last Post
  pbp with manjaro arm seems to not find any hardware[no wifi, blutooth or battery] frank_philip 4 6,343 03-15-2025, 01:47 AM
Last Post: frank_philip
  How to mainline kernel on daniel thompson's debian installer? hellojack 14 23,423 09-07-2023, 09:38 PM
Last Post: Der Geist der Maschine
  kernel module that supports usb-c hdmi alexandre 16 29,865 05-14-2023, 09:58 PM
Last Post: rathofkhan3
  Manjaro update broke hardware video acceleration with mpv myself600 2 5,037 01-25-2023, 09:12 AM
Last Post: myself600
Question Debian (Vanilla) no output on display after Kernel update (6.0.8-1) as365n4 1 4,923 12-09-2022, 12:43 PM
Last Post: as365n4
  bootloader menu to pick a kernel - what is the current situation in in early 2022? maxtothemax 4 8,439 02-05-2022, 03:04 PM
Last Post: maxtothemax
  Kernel panics solostian 17 29,549 01-26-2022, 08:08 AM
Last Post: TRS-80
  Kernel 5.17 will improve VP9 support? jiyong 1 6,001 01-16-2022, 08:26 PM
Last Post: xmixahlx
  red/green power led not working with kernel 5.14 alchemist 4 9,231 11-04-2021, 04:09 AM
Last Post: alchemist
  Very slow glamor 2D acceleration hjalfi 3 9,487 09-06-2021, 12:54 PM
Last Post: jbrock

Forum Jump:


Users browsing this thread: 1 Guest(s)