The state of mainline hardware decoding

[sigmaris]

Code:

-c:v mjpeg_vaapi

This isn't going to get you hardware acceleration on RK3399, this is telling FFmpeg to use the VA-API to talk to the kernel/hardware, but the Hantro encoder driver in the kernel doesn't support this API, it only supports v4l2.

If there was a v4l2-accelerated MJPEG encoder in FFmpeg it would be called "mjpeg_v4l2m2m", but I don't think such an encoder exists, at least I can't see it in my list of ffmpeg -encoders, and I can't find any mention of such a thing online.

I've only seen GStreamer's v4l2jpegenc mentioned as working with the Hantro JPEG encoder, in this thread. I recommend trying the GStreamer encoder (and, the v4l2 stateless codecs e.g. v4l2slh264dec in Gstreamer 1.18 should work for hw-accelerated decoding on RK3399, too)