The state of mainline hardware decoding
Previously I had H.264 decoding working on a 5.10.17 kernel with patches from Pine64 (pbp-tools) for an rk3399.  We had a need to bump to 5.15.14 so I needed to update the decoding stuff, plus get MJPEG encoding working.

I found that the old patches are no longer valid.  The upstreamed rkvdec driver in 5.15.14 works (unpatched) for H.264 decoding using the ffmpeg for xbmc (tag: 4.4-N-Alpha1) plus the LibreELEC patches.  I can decode 4 streams of RTSP, output to null, at 30fps with about 6% CPU usage.  Without hardware decoding this would have been over 25% CPU usage.  The command I used for this test is as shown.

Quote:ffmpeg -y  -hwaccel drm -hwaccel_device /dev/dri/renderD128  -rtsp_transport tcp -i rtsp://  -rtsp_transport tcp -i rtsp://  -rtsp_transport tcp -i rtsp://  -rtsp_transport tcp -i rtsp:// -threads 1 -f null -

I have not been able to get MJPEG encoding working.  This is my test command, which may not be correct.

Quote:ffmpeg -y \
-hwaccel drm \
-hwaccel_device /dev/dri/renderD128 \
-rtsp_transport tcp \
-i rtsp:// \
-c:v mjpeg_vaapi -q:v 10 -an -y -frames 900 -vsync 2 \
-threads 1 -f null -

This command fails.

Quote:Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (mjpeg_vaapi))
Press [q] to stop, [?] for help
[h264 @ 0x19969a10] v4l2_request_try_format: pixelformat 875967059 not supported for type 10
[h264 @ 0x19969a10] v4l2_request_probe_video_device: try output format failed
[h264 @ 0x19969a10] v4l2_request_probe_video_device: set controls failed, Invalid argument (22)
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!

Changing the output codec to mjpeg just gets me a CPU-based encoding, so CPU usage goes up to around 25%.

There is ony one v4l2 JPEG controls on the video1 (and none on video2) device but I'm not sure that implies MJPEG support.

Quote:$ v4l2-ctl --list-devices
rockchip,rk3399-vpu-enc (platform: hantro-vpu):

rkvdec (platform:rkvdec):

$ v4l2-ctl --list-ctrls --device /dev/video1 | grep -i jpeg
JPEG Compression Controls

There is a recent patch for 5.18.x that may improve this but I don't see any additional controls in the patch.  I have not applied the patch or attempted backporting it yet.

If anyone has additional information on the state of mjpeg encoding and/or appropriate tests to use let me know.  Thanks.
Michael J. Hammel

Messages In This Thread
RE: The state of mainline hardware decoding - by mjhammel - 01-26-2022, 12:33 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
Information Linux Mainline ayufan 63 75,245 05-14-2021, 10:41 AM
Last Post: Wizzard
  Hardware acceleration using FFmpeg gusarg81 0 2,335 08-21-2020, 01:36 PM
Last Post: gusarg81
  mainline kernel sound support Openwrt lucize 2 3,162 05-01-2020, 05:09 PM
Last Post: PakoSt
  Hardware Status Monitoring? Leapo 7 12,691 03-15-2020, 08:44 AM
Last Post: bendem
  Any advantages to using the mainline kernel dkebler 0 1,720 11-16-2019, 12:17 PM
Last Post: dkebler
  Does anybody run the mainline kernel? CameronNemo 3 3,732 09-09-2019, 07:56 PM
Last Post: CameronNemo
  hardware-accelerated video transcoding (Plex) on Rock64 mdr 2 5,960 02-07-2019, 03:42 PM
Last Post: mdr
  Updates on Mainline Linux Support? hnaguski 0 2,073 09-11-2018, 08:16 PM
Last Post: hnaguski
  How to do hardware decoding of video? SuperSaiyanCaleb 9 12,795 08-28-2018, 01:39 PM
Last Post: mcerveny
  H264 hardware encoder not work sueshieh 3 5,452 11-02-2017, 03:57 AM
Last Post: dalmate

Forum Jump:

Users browsing this thread: 1 Guest(s)