The state of mainline hardware decoding

[CounterPillow]

Okay, a little update.

Code:

$ v4l2-ctl --list-ctrls --device /dev/video1

Codec Controls

                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=2 value=2

Stateless Codec Controls

               h264_decode_mode 0x00a40900 (menu)   : min=1 max=1 default=1 value=1
                h264_start_code 0x00a40901 (menu)   : min=1 max=1 default=1 value=1
    h264_sequence_parameter_set 0x00a40902 (unknown): type=200 flags=has-payload
     h264_picture_parameter_set 0x00a40903 (unknown): type=201 flags=has-payload
            h264_scaling_matrix 0x00a40904 (unknown): type=202 flags=has-payload
         h264_decode_parameters 0x00a40907 (unknown): type=204 flags=has-payload

Code:

$ gst-launch-1.0 filesrc location=/home/fratti/duckwater.mp4 ! qtdemux ! v4l2slh264dec ! fakesink
Setting pipeline to PAUSED ...
ERROR: from element /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0: Driver did not report framing and start code method.
Additional debug info:
../gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c(134): gst_v4l2_codec_h264_dec_open (): /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0:
gst_v4l2_decoder_get_controls() failed: Invalid argument
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0: Could not initialize supporting library.
Additional debug info:
../gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2666): gst_video_decoder_change_state (): /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0:
Failed to open decoder
ERROR: pipeline doesn't want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

With the DTS from LibreELEC, I got /dev/video1 to show up. It still doesn't work, but this is progress! I first tried to have the dts stuff generated from just the patches, but that somehow didn't work (video-codec for rkvdec never showed up in the resulting .dtb). So I simply built a dtb from their dts that diederik helpfully provided for me, and it's showing the device now.

My guess is that some of the illegal argument stuff is entirely down to userspace not keeping up with kernel changes. That's ok, and I expect this to get resolved once it's moved out of staging (presumably during the 5.14 merge window.)

I have not yet been able to build gst-plugins-bad with the vp8 API changes, but I'll try again over the coming days. Actually seeing it decode would be awesome.

Preliminary conclusion: rk3328 needs some upstream device tree love, but things are looking way brighter than I initially thought they looked.

I need to learn about device trees before I go chase down the "why does this dtsi change not result in a different rk3328-rock64.dtb" issue. I've already been supplied with some resources for this.

A big thank you to everyone both on IRC/Discord/Matrix and the forums who provided help to get me to this point.