+- PINE64 (https://forum.pine64.org)
+-- Forum: Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=111)
+--- Forum: Linux on Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=114)
+--- Thread: mainline kernel hardware acceleration (/showthread.php?tid=9171)
RE: mainline kernel hardware acceleration - xmixahlx - 05-31-2020
linuxtv wip against media_tree master from linuxtv.org
drm next against next20200501 from kernel.org
ffmpeg v4l2 rkvdec against official 4.2.2
i made patchsets against next20200529 preparing for 5.7 to be released. (and prob slight changes afterwards). i was to share them when i get to my pbp (on phone now).
you are right that the patches didn't apply cleanly. i made several changes.
the kernel i am testing now, working well is:
next20200529
pinebookpro patch: https://pastebin.com/YK4LXeyv
LibreElec 1000 patch
v4l2 patch: https://pastebin.com/gaQSZQ6B
drm patch: https://pastebin.com/b1BWZ8Cg
plus:
ffmpeg 4.2.3 with v4l2-request-hwaccel-4.2.2-rkvdec patch: https://pastebin.com/uF5kpABD
kodi-git
libva-v4l2-request_master + pulls 29+30
libva_master + pulls 332+340
(for libva 332 delete the 5th chunk, version bump, or apply against libva 2.6.0)
general responsiveness is actually significantly better than 5.7-rc7 (!?), and my nvme drive is also much better handled. interesting.
also, i would recommend compiling hantro and rkvdec into the kernel, not modules, so that they are detected in the same order consistently.
RE: mainline kernel hardware acceleration - termac - 06-01-2020
Thank You again for Your help @xmixahlx ! Hardware decoding is now working nicely, following the steps You outlined above. Using ffmpeg and null output:
Code:
> ffmpeg -hwaccel drm -i bbb_sunflower_1080p_60fps_normal.mp4 -f null -
ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9 (Debian 9.3.0-13)
configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request --enable-libdrm --enable-libudev --enable-openssl
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb_sunflower_1080p_60fps_normal.mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomavc1
creation_time : 2013-12-16T17:59:32.000000Z
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
genre : Animation
composer : Sacha Goedegebure
Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn, 120 tbc (default)
Metadata:
creation_time : 2013-12-16T17:59:32.000000Z
handler_name : GPAC ISO Video Handler
Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
Metadata:
creation_time : 2013-12-16T17:59:37.000000Z
handler_name : GPAC ISO Audio Handler
Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)
Metadata:
creation_time : 2013-12-16T17:59:37.000000Z
handler_name : GPAC ISO Audio Handler
Side data:
audio service type: main
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Stream #0:2 -> #0:1 (ac3 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: missing required mem2mem capability
Last message repeated 1 times
[h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed
[h264 @ 0xaaaaeaa94de0] v4l2_request_try_format: pixelformat 875967059 not supported for type 10
[h264 @ 0xaaaaeaa94de0] v4l2_request_probe_video_device: try output format failed
frame= 10 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomavc1
composer : Sacha Goedegebure
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
genre : Animation
encoder : Lavf58.29.100
Stream #0:0(und): Video: wrapped_avframe, nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)
Metadata:
creation_time : 2013-12-16T17:59:32.000000Z
handler_name : GPAC ISO Video Handler
encoder : Lavc58.54.100 wrapped_avframe
Stream #0:1(und): Audio: pcm_s16le, 48000 Hz, 5.1(side), s16, 4608 kb/s (default)
Metadata:
creation_time : 2013-12-16T17:59:37.000000Z
handler_name : GPAC ISO Audio Handler
encoder : Lavc58.54.100 pcm_s16le
Side data:
audio service type: main
frame= 52 fps= 52 q=-0.0 size=N/A time=00:00:00.90 bitrate=N/A speed=0.899x
frame= 97 fps= 64 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A speed=1.09x
frame= 141 fps= 70 q=-0.0 size=N/A time=00:00:02.40 bitrate=N/A speed=1.19x
frame= 187 fps= 74 q=-0.0 size=N/A time=00:00:03.15 bitrate=N/A speed=1.24x
frame= 233 fps= 77 q=-0.0 size=N/A time=00:00:03.91 bitrate=N/A speed=1.29x
frame= 283 fps= 80 q=-0.0 size=N/A time=00:00:04.75 bitrate=N/A speed=1.34x
frame= 331 fps= 82 q=-0.0 size=N/A time=00:00:05.55 bitrate=N/A speed=1.37x
frame= 378 fps= 83 q=-0.0 size=N/A time=00:00:06.33 bitrate=N/A speed=1.39x
frame= 406 fps= 83 q=-0.0 Lsize=N/A time=00:00:06.80 bitrate=N/A speed= 1.4x
video:213kB audio:3780kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.So I am already half way there. Now I only need to get video rendering working properly, because the output with mpv is still dropping frames. I did rebuild against patched ffmpeg, but running mpv with drm option for decoding and video output gives me errors and mpv does not play the video:
Code:
$> ~/mpv-build/mpv/build/mpv --hwdec=drm --vo=gpu --gpu-context=drm bbb_sunflower_1080p_60fps_normal.mp4
(+) Video --vid=1 (*) (h264 1920x1080 60.000fps)
(+) Audio --aid=1 (*) (mp3 2ch 48000Hz)
Audio --aid=2 (*) (ac3 6ch 48000Hz)
File tags:
Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
Composer: Sacha Goedegebure
Genre: Animation
Title: Big Buck Bunny, Sunflower version
[vo/gpu] VT_GETMODE failed: Inappropriate ioctl for device
[vo/gpu/opengl] Failed to set up VT switcher. Terminal switching will be unavailable.
[vo/gpu] Failed to commit ModeSetting atomic request (-13)
[vo/gpu/opengl] Failed to set CRTC for connector 49: Permission denied
[vo/gpu] Failed initializing any suitable GPU context!
Error opening/initializing the selected video_out (--vo) device.
Video: no video
Exiting... (Errors when loading file)Leaving off the "--gpu-context=drm" option will allow the file to play, but as I said output is dropping frames.
Do You happen to know what the issue here is? From Your posts I gather, You do not use mpv Yourself, but maybe You have come across something... In any case thank You very much again, this is already way better, than what I had working before!
RE: mainline kernel hardware acceleration - xmixahlx - 06-01-2020
i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)
for quick test monitor top and use: "ffmpeg -re --hwaccel drm -i inputfile -f null -"
for hwaccel playback use kodi-git. see my other thread. i am also using jellyfin-kodi. works great playing files from my htpc/server.
5.7 is released so i will update patches soon.
RE: mainline kernel hardware acceleration - termac - 06-01-2020
(06-01-2020, 09:19 AM)xmixahlx Wrote: i think mpv hwaccel is currently broken due to v4l2request not being detected. (ffmpeg -decoders | grep v4l)
...
Code:
$> ffmpeg -decoders | grep v4l
ffmpeg version 4.2.3-Kodi Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9 (Debian 9.3.0-13)
configuration: --prefix=/usr/local --libdir=/usr/local/lib/aarch64-linux-gnu --shlibdir=/usr/local/lib/aarch64-linux-gnu --enable-shared --enable-pic --enable-v4l2-request
--enable-libdrm --enable-libudev --enable-openssl
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
V..... h263_v4l2m2m V4L2 mem2mem H.263 decoder wrapper (codec h263)
V..... h264_v4l2m2m V4L2 mem2mem H.264 decoder wrapper (codec h264)
V..... hevc_v4l2m2m V4L2 mem2mem HEVC decoder wrapper (codec hevc)
V..... mpeg1_v4l2m2m V4L2 mem2mem MPEG1 decoder wrapper (codec mpeg1video)
V..... mpeg2_v4l2m2m V4L2 mem2mem MPEG2 decoder wrapper (codec mpeg2video)
V..... mpeg4_v4l2m2m V4L2 mem2mem MPEG4 decoder wrapper (codec mpeg4)
V..... vc1_v4l2m2m V4L2 mem2mem VC1 decoder wrapper (codec vc1)
V..... vp8_v4l2m2m V4L2 mem2mem VP8 decoder wrapper (codec vp8)
V..... vp9_v4l2m2m V4L2 mem2mem VP9 decoder wrapper (codec vp9)Edit: Isn't that the expected output? I see now all the *_v4l2request are missing. Sorry!
The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.
RE: mainline kernel hardware acceleration - spikerguy - 06-01-2020
Hi @xmixahlx
I would like to know where did you get these patches from ?
Is there any plan to upstream these?
I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?
I would love to test it too and build, package it for manjaro
Keep up the good work.
RE: mainline kernel hardware acceleration - xmixahlx - 06-01-2020
(06-01-2020, 09:24 AM)termac Wrote: The ffmpeg comand you suggested uses ~75% of one core and decodes at 60 fps for a 1080p/60fps file. So I guess it works in principal. Will have to test kodi next than.
for a better example of the kodi-gbm+drm+gles experience, use: "ffmpeg -re --hwaccel drm -hwaccel_output_format drm_prime -i inputfile -f null -"
you can also add pixel format with "-pix_fmt bgra" and output to framebuffer with "-f fbdev /dev/fb0"
(06-01-2020, 02:31 PM)spikerguy Wrote: Hi @xmixahlx
I would like to know where did you get these patches from ?
Is there any plan to upstream these?
I understand we need v4l2 in ffmpeg but isn't the drm already available in 5.7 ?
I would love to test it too and build, package it for manjaro
Keep up the good work.
@spikerguy @tsys
the drm is available in 5.7, but it is patched for upstream + WIP features. the v4l2 api is in constant flux. kwiboo upstreams some patches, and what's still WIP is included in libreelec. their build system pulls in patches and builds packages from source, which makes adapting their patches from their git repo easier than cherry picking patches from LKML (!!!).
in more detail: after a major release he updates his repo branches and gets ready to package for libreelec, then he makes a PR on the libreelec main git. when merged he deletes his branch.
you can generally make his patches yourself by diffing each branch to their base. (that's what i do between his major releases.)
a few additional WIP things are not included in the official patches. you can get those via the branch diffs.
https://patchwork.kernel.org/project/linux-media/list/?submitter=182393
https://github.com/Kwiboo/linux-rockchip
https://github.com/Kwiboo/LibreELEC.tv/tree/rk-5.7/projects/Rockchip/patches/linux/default
for packaging, you would also want to include ffmpeg-v4l2-request. (the diff against official 4.2.2 is above and applies to 4.2.3, also), and kodi-git built with gbm+gles (see my other thread).
ffmpeg-v4l2-request: https://github.com/Kwiboo/FFmpeg/tree/v4l2-request-hwaccel-4.2.2
ffmpeg-v4l2-request-rockchip: https://github.com/Kwiboo/FFmpeg/tree/v4l2-request-hwaccel-4.2.2-rkvdec
(have these conflict. rkvdec is experimental for rockchip, the other is stable for both rockchip and allwinner, but less featureful on rockchip.)
libva and libva-v4l2-request are somewhat broken at the moment and can be skipped for packaging right now.
edit: i attached my patches, config, and patchseries script (delete .log). you can see several of these made it into libreelec 5.7 branch with similar naming.
RE: mainline kernel hardware acceleration - termac - 06-03-2020
Thanks for the patches and kernel config @xmixahlx !
Trying to build gives me an error regarding firmware:
Code:
make KERNELRELEASE=5.7.0-pbp-custom ARCH=arm64 KBUILD_BUILD_VERSION=1 -f ./Makefile
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
CHK include/generated/compile.h
make[7]: *** No rule to make target '/lib/firmware/rockchip/dptx.bin', needed by 'drivers/base/firmware_loader/builtin/rockchip/dptx.bin.gen.o'. Stop.
make[6]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader/builtin] Error 2
make[5]: *** [scripts/Makefile.build:488: drivers/base/firmware_loader] Error 2
make[4]: *** [scripts/Makefile.build:488: drivers/base] Error 2
make[3]: *** [Makefile:1729: drivers] Error 2
make[2]: *** [debian/rules:6: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 2
make: *** [Makefile:1460: bindeb-pkg] Error 2Anyone got an idea if the file '/lib/firmware/rockchip/dptx.bin' is really needed (it is not present in my earlier kernel build trees) or what to do here?
RE: mainline kernel hardware acceleration - xmixahlx - 06-03-2020
i build this into the kernel, but you can remove by deleting that line in the config. it is installed manually by danielt's installer, and included in debian's firmware-misc-nonfree package.
RE: mainline kernel hardware acceleration - termac - 06-03-2020
Installing firmware-misc-nonfree did the trick. Thanks again!
RE: mainline kernel hardware acceleration - xmixahlx - 06-07-2020
confirmed that the youtube kodi plugin working well with hwaccel (tested up to 1080p). h264 works great. vp9 has occasional artifacts. recommend using Adaptive H264 in MPEG DASH settings.
kodi-git thread updated and added inputstream.adaptive-git script for use with youtube-kodi-addon, etc.