mainline kernel hardware acceleration
#1
collecting information and projects regarding hardware acceleration in mainline kernel, ffmpeg, and vaapi.

STATUS:
drm hwaccel working with hantro (mpeg2, vp8, h264) and rkvdec (vp9, h264 8+10bit, h265 8+10bit)
vaapi hwaccel working with hantro (mpeg2)
firefox 75 supports initial vaapi on wayland, v76 expands vaapi support to all supported codecs, no X11 support
chrome supports vaapi with unofficial patch that will never be upstreamed, but included in some linux distros (fedora, etc.)

REFERENCES

linux patchwork - linux-media:
https://patchwork.kernel.org/project/linux-media/list/

linux patchwork - linux-rockchip:
https://patchwork.kernel.org/project/lin...chip/list/

libreelec/kwiboo mainline kernel patches:
https://github.com/LibreELEC/LibreELEC.t...ux/default

libreelec/kwiboo mainline kernel testing patches:
https://github.com/Kwiboo/LibreELEC.tv

kwiboo's rockchip-linux repos:
https://github.com/Kwiboo/linux-rockchip/

bbrezillon's rockchip-linux repos:
https://github.com/bbrezillon/linux/

ph5's libva patch:
https://github.com/intel/libva/pull/332

libva-v4l2-request;
https://github.com/bootlin/libva-v4l2-request/

libva-v4l2-request WIP h265 work:
https://github.com/bootlin/libva-v4l2-re...ad6531cd97

ph5's libva-v4l2-request patches:
https://github.com/bootlin/libva-v4l2-request/pulls

kwiboo's ffmpeg v4l2-request branch:
https://github.com/Kwiboo/FFmpeg/tree/v4...ccel-4.2.2
https://github.com/Kwiboo/FFmpeg/tree/v4...2.2-rkvdec

gstreamer vaapi:
https://gitlab.freedesktop.org/gstreamer...quests/171

firefox vaapi:
https://bugzilla.mozilla.org/show_bug.cgi?id=1610199

chrome vaapi:
https://chromium-review.googlesource.com...c/+/532294
https://github.com/saiarcot895/chromium-...an/patches
https://aur.archlinux.org/packages/chromium-vaapi/

OTHER

allwinner cedrus hwaccel project:
https://linux-sunxi.org/Sunxi-Cedrus

Arch - hardware acceleration:
https://wiki.archlinux.org/index.php/Har...celeration

LibreElec - Early Mainline Images for <...> RK3399:
https://forum.libreelec.tv/thread/17540-...nd-rk3399/

LibreElec - Not Official LE Version for <...> RK3399
https://forum.libreelec.tv/thread/20823-...28-rk3399/

Armbian - early VPU work:
https://forum.armbian.com/topic/4806-the-vpu-driver/

Armbian - Mainline VPU
https://forum.armbian.com/topic/13622-mainline-vpu/

miouyouyou's early build scripts:
https://github.com/Miouyouyou/RockMyy

miouyouyou's ffmpeg v4l2-request patches:
https://github.com/Miouyouyou/FFmpeg-V4L2-Request
  Reply
#2
mpeg2video hwaccel is tested working.

testing requirements:
patched kernel (libreelec; optionally kwiboo v4l2 5.5 wip and rkvdec, plus bbrezillon patches)
patched libva (plus pulls 332,340)
patched libva-v4l2-request (plus pulls 28,29,30)
patched ffmpeg (v4l2-request-hwaccel) -- recommend 4.2.2 or -master
> build with --enable-v4l2-request --enable-libdrm --enable-libudev --enable-pic --enable-shared (plus optional config)

determine video decoder:
dmesg | grep rk3399-vpu-dec
note video device (i.e. /dev/video1)

export variables:
export LIBVA_DRIVER_NAME=v4l2_request
export LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1

vainfo output:
Quote:libva info: VA-API version 1.6.1
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/local/lib/aarch64-linux-gnu/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.6 (libva 2.6.0)
vainfo: Driver version: v4l2-request
vainfo: Supported profile and entrypoints
     VAProfileMPEG2Simple            : VAEntrypointVLD
     VAProfileMPEG2Main              : VAEntrypointVLD
     VAProfileH264Main               : VAEntrypointVLD
     VAProfileH264High               : VAEntrypointVLD
     VAProfileH264ConstrainedBaseline: VAEntrypointVLD
     VAProfileH264MultiviewHigh      : VAEntrypointVLD
     VAProfileH264StereoHigh         : VAEntrypointVLD

vlc abbreviated output:
> opengl video, vaapi hwaccel
Quote:libva info: VA-API version 1.6.1
libva info: User environment variable requested driver 'v4l2_request'
libva info: Trying to open /usr/local/lib/aarch64-linux-gnu/dri/v4l2_request_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
[0000ffff5cc19640] avcodec decoder: Using v4l2-request for hardware decoding

ffmpeg -decoders | grep v4l2request output:
Quote: V..... h263_v4l2request h263 (v4l2request) (codec h263)
V..... h264_v4l2request h264 (v4l2request) (codec h264)
V..... hevc_v4l2request hevc (v4l2request) (codec hevc)
V..... mpeg1_v4l2request mpeg1 (v4l2request) (codec mpeg1video)
V..... mpeg2_v4l2request mpeg2 (v4l2request) (codec mpeg2video)
V..... mpeg4_v4l2request mpeg4 (v4l2request) (codec mpeg4)
V..... vc1_v4l2request vc1 (v4l2request) (codec vc1)
V..... vp8_v4l2request vp8 (v4l2request) (codec vp8)
V..... vp9_v4l2request vp9 (v4l2request) (codec vp9)

screenshot eye candy
  Reply
#3
@xmixahlx  great work, I was looking for something like this to as I was going to test Kodi on my PBP with but with VPU support and using USB-C to DVI (HDMI) on Manjaro KDE. But seems there is no already package for these kinde of support on Manjaro KDE jet.

Do you have some sources for your PKGBUILDs or you are building packages without PKGBUILDs?

From arbian forum I have these information (from 2019), the kwiboo's rockchip-linux repos you already linked.
Code:
The VPU in rk3288 (vpu1) should have VP8 support, same as the VPU in rk3328/rk3399 (vpu2)
rk3288 has a second VPU block that can decode hevc
rk3328/rk3399 has a second VPU block (rkvdec) that can decode h264/hevc/vp9 (they have two different vpu blocks that can decode h264)
  Reply
#4
(02-19-2020, 12:38 AM)vrabac Wrote: @xmixahlx  great work, I was looking for something like this to as I was going to test Kodi on my PBP with but with VPU support and using USB-C to DVI (HDMI) on Manjaro KDE. But seems there is no already package for these kinde of support on Manjaro KDE jet.

Do you have some sources for your PKGBUILDs or you are building packages without PKGBUILDs?

From arbian forum I have these information (from 2019), the kwiboo's rockchip-linux repos you already linked.
Code:
The VPU in rk3288 (vpu1) should have VP8 support, same as the VPU in rk3328/rk3399 (vpu2)
rk3288 has a second VPU block that can decode hevc
rk3328/rk3399 has a second VPU block (rkvdec) that can decode h264/hevc/vp9 (they have two different vpu blocks that can decode h264)

i'm on debian sid arm64 and following guidance from kernel patches and commentary. keep an eye out for av's arch patches. we're collaborating on pine64 irc occasionally.
https://aur.archlinux.org/packages/?O=0&..._Search=Go
  Reply
#5
Fantastic! I haven't been able to replace my Thinkpad just yet because of a lack of video acceleration in Manjaro. Looks like it won't be too long until it'll be plug and play.
  Reply
#6
(02-19-2020, 08:20 AM)amiraeva Wrote: Fantastic! I haven't been able to replace my Thinkpad just yet because of a lack of video acceleration in Manjaro. Looks like it won't be too long until it'll be plug and play.

The last blog post had a panfrost installation script that got hardware acceleration working on my pine book pro.
  Reply
#7
panfrost hwaccel and hantro/rkvdec hwaccel are different.

hantro/rkvdec are the rockchip hwaccel kernel modules for media decoding.

there are numerous capabilities of the rockchip CPU that are not yet utilized, and development is ongoing.

one of the main developers, kwiboo, also is on the libreelec team and their distro is the most advanced in this regard. other distros will use this as a base until accepted upstream to provide similar support.
  Reply
#8
is the patched kernel  an must to or optional? I used under Manjaro Arm 20.02 the AndreVallestero PKGBUILDs but having problem to compile ffmpeg but i just compiled and installed these to prior to compile ffmpeg: libva-hantro-h264-git and libva-v4l2-request-hantro-h264-git

Code:
CC      libavcodec/v4l2_m2m.o
In file included from /usr/include/string.h:494,
                from ./libavutil/common.h:40,
                from ./libavutil/avutil.h:296,
                from ./libavutil/samplefmt.h:24,
                from ./libavcodec/avcodec.h:31,
                from libavcodec/v4l2_m2m.c:30:
In function ‘strncpy’,
   inlined from ‘ff_v4l2_m2m_codec_init’ at libavcodec/v4l2_m2m.c:362:9:
/usr/include/bits/string_fortified.h:106:10: warning: ‘__builtin___strncpy_chk’ specified bound depends on the length of the source argument [-Wstringop-overflow=]
 106 |   return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
     |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libavcodec/v4l2_m2m.c: In function ‘ff_v4l2_m2m_codec_init’:
libavcodec/v4l2_m2m.c:362:35: note: length computed here
 362 |         strncpy(s->devname, node, strlen(node) + 1);
     |                                   ^~~~~~~~~~~~
CC      libavcodec/v4l2_m2m_dec.o
CC      libavcodec/v4l2_m2m_enc.o
CC      libavcodec/v4l2_request.o
libavcodec/v4l2_request.c: In function ‘v4l2_request_queue_decode’:
libavcodec/v4l2_request.c:216:90: error: ‘V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF’ undeclared (first use in this function)
 216 |     ret = v4l2_request_queue_buffer(ctx, req->request_fd, &req->output, last_slice ? 0 : V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF);
     |                                                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libavcodec/v4l2_request.c:216:90: note: each undeclared identifier is reported only once for each function it appears in
libavcodec/v4l2_request.c: In function ‘ff_v4l2_request_decode_slice’:
libavcodec/v4l2_request.c:300:37: error: ‘V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF’ undeclared (first use in this function)
 300 |     if ((req->output.capabilities & V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF) != V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF)
     |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [ffbuild/common.mak:59: libavcodec/v4l2_request.o] Error 1
==> ERROR: A failure occurred in build().
   Aborting...
  Reply
#9
the patched kernel provides necessary kernel headers for ffmpeg-v4l2-request, although mpeg2video hwaccel is now working with mainline kernel. (i tested with 5.6.0-rc3+tsys). i recommend applying the libreelec kernel patches if you want to test.

things are picking up momentum. it looks like 5.7 will include many hwaccel improvements. see:
https://patchwork.kernel.org/patch/11403025/
  Reply
#10
a good read on firefox wayland:
https://mastransky.wordpress.com/2020/03...n-wayland/
  Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  kodi-git build script for mainline hwaccel media decoding xmixahlx 3 191 05-21-2020, 09:07 PM
Last Post: xmixahlx
  A true mainline Linux Kernel for the Pinebook Pro tsys 138 21,466 04-22-2020, 09:05 AM
Last Post: xmixahlx
  dim screen after kernel upgrade bsammon 4 182 03-22-2020, 10:29 PM
Last Post: xmixahlx
  The default OS / kernel had a wifi issue? dawncold 15 1,239 03-02-2020, 06:26 PM
Last Post: lowry
  Booting with modified kernel parameters? zackw 1 237 02-29-2020, 10:14 AM
Last Post: Arwen
  Manjaro with BSP kernel jichu4n 0 202 02-22-2020, 04:35 PM
Last Post: jichu4n
  Kernel headers for stock Debian image fader 0 108 01-18-2020, 10:35 AM
Last Post: fader
  Troubleshooting New Kernel Image wsgts 3 232 01-16-2020, 09:04 AM
Last Post: wsgts
  HW Acceleration for 2d/3d on pinebook pro ljones 1 758 01-02-2020, 02:04 PM
Last Post: ElektromAn
  Hardware acceleration on PBP brent.thierens 9 973 12-23-2019, 10:31 AM
Last Post: ElektromAn

Forum Jump:


Users browsing this thread: xmixahlx, 1 Guest(s)