This article gives information about the Linux® V4L2 video codec framework.
1. Framework purpose[edit | edit source]
The V4L2 Linux kernel framework[1] allows the control of video hardware codecs to decode and encode compressed video contents such as H264, VP8, or JPEG video bitstreams.
This could be typically used, with the help of other Linux multimedia frameworks and applications, to play local videos, stream video contents, make a video recording, or even remotely stream video compressed images from the camera sensor.
2. System overview[edit | edit source]
2.1. Component description[edit | edit source]
- Application (user space)
Any application relying on the GStreamer framework or V4L2 Linux kernel interface[2].
- gst-play (user space)
Gst-play is a GStreamer command-line utility playing multimedia content using the playbin element as a basic player.
- playbin (user space)
playbin is a complex GStreamer bin playing multimedia content using the decodebin element to abstract video, audio, or metadata (subtitles) decoding process.
- decodebin (user space)
decodebin is a complex GStreamer bins abstracting hardware acceleration to user and automatically selecting the best level of performances.
- v4l2slh264dec (user space)
A GStreamer element in charge of H264 bitstream decoding using a V4L2 hardware accelerated decoder driver such as the hantro_vpu kernel driver. "sl" stands for "stateless" which means that compressed bitstream is processed by user space GStreamer v4l2codecs libraries to generate the decoding metadata required by V4L2 kernel driver.
- v4l2slvp8dec (user space)
A GStreamer element in charge of VP8 elementary bitstream decoding using a V4L2 hardware accelerated decoder driver such as the hantro_vpu kernel driver.
- v4l2jpegdec (user space)
A GStreamer element in charge of JPEG decoding using a V4L2 hardware accelerated decoder driver such as the hantro_vpu kernel driver.
- gst-transcoder (user space)
Gst-transcoder is a GStreamer command-line utility transcoding multimedia content using decodebin and encodebin complex bins.
- encodebin (user space)
encodebin is a complex GStreamer bins abstracting hardware acceleration to user and automatically selecting the best level of performances.
- v4l2slh264enc (user space)
A GStreamer element in charge of H264 elementary bitstream encoding using a V4L2 hardware accelerated encoder driver such as the hantro_vpu kernel driver. "sl" stands for "stateless" which means that the compressed bitstream metadata are generated by user space GStreamer v4l2codecs libraries on top of the compressed content provided by V4L2 kernel driver.
- v4l2slvp8enc (user space)
A GStreamer element in charge of VP8 elementary bitstream encoding using a V4L2 hardware accelerated decoder driver such as the hantro_vpu kernel driver.
- v4l2jpegenc (user space)
A GStreamer element in charge of JPEG encoding using a V4L2 hardware accelerated decoder driver such as the hantro_vpu kernel driver.
- V4L2 core (kernel space)
This layer represents the standard Linux kernel V4L2 framework.
- hantro_vpu (kernel space)
This V4L2 Linux device driver handles the VDEC hardware block and the VENC hardware block.
- VDEC (hardware)
The Video DECoder hardware block.
- VENC (hardware)
The Video ENCoder hardware block.
2.2. APIs description[edit | edit source]
The V4L2 userland API is documented in the Linux media subsystem documentation[2]
More specifically, the V4L2 video memory-to-memory userland API is documented in the chapter[3]
The V4L2 kernel framework internal API is documented in the V4L2 kernel support section of the Linux kernel documentation[4]
The media controller API is documented in the Linux media subsystem documentation[5]
3. Configuration[edit | edit source]
3.1. Kernel configuration[edit | edit source]
The STM32 hantro_vpu driver is enabled by default in STMicroelectronics deliveries.
Nevertheless, this could not be the case when using the upstream kernel version. In this case, the hantro_vpu V4L2 driver with its STM32MP25 back-end can be enabled using Linux kernel menuconfig tool:
[*] Device Drivers --->
[*] Multimedia support --->
[*] Media drivers --->
[*] Media platform devices --->
[*] Hantro VPU driver --->
[*] Hantro STM32MP25 support
3.2. Device tree configuration[edit | edit source]
Refer to the VDEC device tree configuration and the VENC device tree configuration articles for Linux kernel device tree details.
4. How to use the framework[edit | edit source]
The use cases described here are enabled using media-ctl, v4l2-ctl, gst-launch, or gst-play command-line utilities.
4.1. List the video codec devices and their capabilities[edit | edit source]
List all the available video devices using the v4l2-ctl --list-devices option:
v4l2-ctl --list-devices
st,stm32mp25-vdec-dec (platform: hantro-vpu):
/dev/video0
/dev/media0
st,stm32mp25-venc-enc (platform: hantro-vpu):
/dev/video1
/dev/media1
If several devices are available, use the -d option after any v4l2-ctl commands to target a specific device. If the -d option is not specified, /dev/video0 is targeted by default.
To have information on a specific device, use the -D option.
/dev/video0 is the video decoder device:
v4l2-ctl -d /dev/video0 -D
Driver Info:
Driver name : hantro-vpu
Card type : st,stm32mp25-vdec-dec
Bus info : platform: hantro-vpu
Driver version : 6.1.28
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : hantro-vpu
Model : hantro-vpu
Serial :
Bus info : platform: hantro-vpu
Media version : 6.1.28
Hardware revision: 0x00000000 (0)
Driver version : 6.1.28
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : st,stm32mp25-vdec-dec-source
Function : V4L2 I/O
Pad 0x01000002 : 0: Source
Link 0x02000008: to remote pad 0x1000004 of entity 'st,stm32mp25-vdec-dec-proc' (Video Decoder): Data, Enabled, Immutable
/dev/video1 is the encoder device:
v4l2-ctl -d /dev/video1 -D
Driver Info:
Driver name : hantro-vpu
Card type : st,stm32mp25-venc-enc
Bus info : platform: hantro-vpu
Driver version : 6.1.28
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : hantro-vpu
Model : hantro-vpu
Serial :
Bus info : platform: hantro-vpu
Media version : 6.1.28
Hardware revision: 0x00000000 (0)
Driver version : 6.1.28
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : st,stm32mp25-venc-enc-source
Function : V4L2 I/O
Pad 0x01000002 : 0: Source
Link 0x02000008: to remote pad 0x1000004 of entity 'st,stm32mp25-venc-enc-proc' (Video Encoder): Data, Enabled, Immutable
4.2. Get the supported compressed format, pixel format, and resolution[edit | edit source]
Use the v4l2-ctl --list-formats-out-ext option to get the compressed formats and resolutions supported by the decoder:
v4l2-ctl --list-formats-out-ext ioctl: VIDIOC_ENUM_FMT [0]: 'VP8F' (VP8 Frame, compressed) Size: Stepwise 48x48 - 1920x1088 with step 16/16 [1]: 'S264' (H.264 Parsed Slice Data, compressed) Size: Stepwise 48x48 - 1920x1088 with step 16/16
Both VP8 and H264 bitstream decoding are supported up to HD at 60 fps.
Use --list-formats-ext option to get the supported output pixel formats:
v4l2-ctl -d /dev/video0 --list-formats-ext ioctl: VIDIOC_ENUM_FMT [0]: 'NV12' (Y/UV 4:2:0)
Only YUV420 semi-planar is supported in output of decoder.
Informations are also available at the GStreamer level by inspecting v4l2slvp8dec and v4l2slh264dec decoder elements using the commands:
gst-inspect-1.0 --no-colors v4l2slvp8dec [...] Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-vp8 SRC template: 'src' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)YUY2, (string)NV12_4L4, (string)NV12_32L32, (string)NV12_16L32S, (string)I420, (string)P010_10LE, (string)RGB16, (string)BGRx, (string)RGBx, (string)xBGR, (string)xRGB } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] [...]
gst-inspect-1.0 --no-colors v4l2slh264dec [...] Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-h264 SRC template: 'src' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)YUY2, (string)NV12_4L4, (string)NV12_32L32, (string)NV12_16L32S, (string)I420, (string)P010_10LE, (string)RGB16, (string)BGRx, (string)RGBx, (string)xBGR, (string)xRGB } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] [...]
The same can be applied for the encoder:
v4l2-ctl -d /dev/video1 --list-formats-out-ext ioctl: VIDIOC_ENUM_FMT [0]: 'NV12' (Y/UV 4:2:0) [1]: 'YM12' (Planar YUV 4:2:0 (N-C)) [2]: 'NM12' (Y/UV 4:2:0 (N-C)) [3]: 'YUYV' (YUYV 4:2:2) [4]: 'UYVY' (UYVY 4:2:2) [5]: 'RGBP' (16-bit RGB 5-6-5) [6]: 'XR24' (32-bit BGRX 8-8-8-8) [7]: 'BGR4' (32-bit BGRA/X 8-8-8-8) [8]: 'XB24' (32-bit RGBX 8-8-8-8) [9]: 'RX24' (32-bit XBGR 8-8-8-8) [10]: 'BX24' (32-bit XRGB 8-8-8-8) [11]: 'RGB4' (32-bit A/XRGB 8-8-8-8)
Encoder can be fed with YUV420 semi-planar, YUV420 planar, YUV422 coplanar, RGB565 16 bits, and RGB 32 bits pixels formats.
v4l2-ctl -d /dev/video1 --list-formats-ext [0]: 'JPEG' (JFIF JPEG, compressed) Size: Stepwise 96x96 - 4096x2304 with step 16/16 [1]: 'VP8F' (VP8 Frame, compressed) Size: Stepwise 96x96 - 4096x2304 with step 16/16 [2]: 'S264' (H.264 Parsed Slice Data, compressed) Size: Stepwise 96x96 - 4096x2304 with step 16/16
Encoder can generate JPEG, VP8, and H264 bitstream up to 4096x2304 but with a maximum performance of HD at 60fps.
Informations are also available at the GStreamer level by inspecting v4l2jpegenc, v4l2slvp8enc and v4l2slh264enc encoder elements using the command:
gst-inspect-1.0 --no-colors v4l2jpegenc [...] Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)I420, (string)YUY2, (string)UYVY, (string)RGB16, (string)BGRA, (string)RGBx, (string)xBGR, (string)ARGB, (string)xRGB, (string)BGRx, (string)NV12 } width: [ 1, 32768 ] height: [ 1, 32768 ] framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: image/jpeg parsed: true [...]
gst-inspect-1.0 --no-colors v4l2slvp8enc [...] Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)YUY2, (string)NV12_4L4, (string)NV12_32L32, (string)NV12_16L32S, (string)I420, (string)P010_10LE, (string)RGB16, (string)BGRx, (string)RGBx, (string)xBGR, (string)xRGB } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: video/x-vp8 [...]
gst-inspect-1.0 --no-colors v4l2slh264enc [...] Pad Templates: SINK template: 'sink' Availability: Always Capabilities: video/x-raw format: { (string)NV12, (string)YUY2, (string)NV12_4L4, (string)NV12_32L32, (string)NV12_16L32S, (string)I420, (string)P010_10LE, (string)RGB16, (string)BGRx, (string)RGBx, (string)xBGR, (string)xRGB } width: [ 1, 2147483647 ] height: [ 1, 2147483647 ] framerate: [ 0/1, 2147483647/1 ] SRC template: 'src' Availability: Always Capabilities: video/x-h264 [...]
4.3. Playing/streaming a video[edit | edit source]
When using high level utilities such as gst-play or complex GStreamer bins such as playbin, playbin3, decodebin, or decodebin3, hardware acceleration is abstracted to the user and automatically selected to achieve the best level of performances.
Refer to the gst-play GStreamer command-line utility for examples of basic audio/video playback or HTTP streaming.
4.4. Encoding a video[edit | edit source]
When using complex GStreamer bins such as encodebin or encodebin2, hardware acceleration is abstracted to the user and automatically selected to achieve the best level of performances.
Here is an example with a local encoding of a 10 s video test pattern in VP8:
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! matroskamux ! filesink location=v_vp8_640x480_30fps.webm
[...]
Got EOS from element "pipeline0".
Execution ended after 0:00:01.229040368
[...]
Play back the file to check the encoded content:
gst-play-1.0 v_vp8_640x480_30fps.webm [...] 0:00:10.0 / 0:00:10.0 Reached end of play list.
Here is another example with a local encoding of a 10 s video test pattern in H264:
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264" ! qtmux ! filesink location=v_h264_640x480_30fps.mp4
[...]
Got EOS from element "pipeline0".
Execution ended after 0:00:01.125719575
[...]
Play back the file to check the encoded content:
gst-play-1.0 v_h264_640x480_30fps.mp4 [...] 0:00:10.0 / 0:00:10.0 Reached end of play list.
 Information
|
| Note that encoding is done as fast as possible, about 1.2 s here but play back of encoded content well shows 10 s of videos played meaning that the framerate information is well considered while encoding. |
4.5. Controlling encoder[edit | edit source]
It is interesting to tune quality, bitrate, and I frames refresh to better fit to the application requirements. Local video recording from a camera requires a good image quality with variable bitrate to adapt to the image complexity while video streaming or visiophony requires bandwidth respect with a constant bitrate dealing with quality tradeoff.
There are no common controls yet at encodebin level to tune quality, bitrate, and key frames refresh rate. To do so, the encoder element must be directly controlled:
gst-inspect-1.0 --no-colors v4l2slvp8enc
[...]
Element Properties:
bitrate : Set bitrate target
flags: readable, writable
Unsigned Integer64. Range: 0 - 4294967295 Default: 100000
keyframe-interval : Interval between keyframes
flags: readable, writable
Integer. Range: 0 - 2147483647 Default: 30
max-quality : Set upper quality limit (lower number equates to higher quality but more bits)
flags: readable, writable
Integer. Range: 0 - 63 Default: 63
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
min-quality : Set lower quality limit (lower number equates to higher quality but more bits)
flags: readable, writable
Integer. Range: 0 - 63 Default: 0
rotation : Set rotation angle in degrees
flags: readable, writable
Unsigned Integer. Range: 0 - 270 Default: 0
gst-inspect-1.0 --no-colors v4l2slh264enc
Element Properties:
bitrate : Set the targeted bitrate (in bit/s)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
cabac : Enable Cabac
flags: readable, writable
Boolean. Default: false
cabac-init-idc : Set Cabac init idc value
flags: readable, writable
Unsigned Integer. Range: 0 - 2 Default: 0
keyframe-interval : Maximum distance in frames between IDR.
flags: readable, writable
Integer. Range: 0 - 2147483647 Default: 30
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
qp-max : Set upper qp limit (lower number equates to higher quality but more bits)
flags: readable, writable
Integer. Range: 0 - 51 Default: 51
qp-min : Set lower qp limit (lower number equates to higher quality but more bits)
flags: readable, writable
Integer. Range: 0 - 51 Default: 10
qp-step : Set maximum value which qp value can be increase/decrease by the bitrate controller (Valid only with rate-control=cbr)
flags: readable, writable
Integer. Range: 0 - 51 Default: 4
quantizer : Set the qp value (lower number equates to higher quality but more bits, initial value for rate-control=cbr)
flags: readable, writable
Integer. Range: 0 - 51 Default: 18
rate-control : Select rate control mode
flags: readable, writable
Enum "GstRateControlMode" Default: 0, "Constant QP"
(0): Constant QP - cqp
(1): Constant Bitrate - cbr
rotation : Set rotation angle in degrees
flags: readable, writable
Unsigned Integer. Range: 0 - 270 Default: 0
Nevertheless, encodebin can still be used while tuning the encoder with a custom "profile=<video> | element-properties,<property=value>" property, some examples are given in below chapters.
| Information
|
| Refer to encoding-profile.c for detailed explanations on how to set the encodebin "profile" property. |
4.5.1. Controlling encoder quality[edit | edit source]
For quality controls -min/max-quality, qp-min/max- the lower the value, the better the quality.
Here are some examples to tune the VP8 quality:
#best quality gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8|element-properties,min-quality=0,max-quality=0" ! matroskamux ! filesink location=v_vp8_640x480_30fps_best.webm
#worst quality gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8|element-properties,min-quality=63,max-quality=63" ! matroskamux ! filesink location=v_vp8_640x480_30fps_worst.webm
ls -s v_vp8_640x480_30fps_best.webm v_vp8_640x480_30fps_worst.webm 14132 v_vp8_640x480_30fps_best.webm 3804 v_vp8_640x480_30fps_worst.webm
We can see that quality impacts by four times the size of the encoded video.
Here are some examples to tune the H264 quality:
#best quality gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,qp-min=0,qp-max=0" ! qtmux ! filesink location=v_h264_640x480_30fps_best.mp4
#worst quality gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,qp-min=51,qp-max=51" ! qtmux ! filesink location=v_h264_640x480_30fps_worst.mp4
ls -s v_h264_640x480_30fps_best.mp4 v_h264_640x480_30fps_worst.mp4 10864 v_h264_640x480_30fps_best.mp4 500 v_h264_640x480_30fps_worst.mp4
We can see that quality impacts by 20 times the size of the encoded video.
4.5.2. Controlling encoder bitrate[edit | edit source]
4.5.2.1. Variable bitrate[edit | edit source]
By default, the encoder is in variable bitrate mode (VBR). Bitrate is adjusted to fit input image complexity, resulting in a good tradeoff between quality and size. Input image dimensions and profile/level are also considered.
Here is an example with H264 VGA at 30 fps:
gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264" ! qtmux ! filesink location=v_h264_640x480_30fps_vbr.mp4
ls -al v_h264_640x480_30fps_vbr.mp4 -rw-r--r-- 1 root root 6072003 May 30 23:31 v_h264_640x480_30fps_vbr.mp4
Meaning 4.8 Mb/s for H264 VGA at 30 fps.
Here is another example with H264 HD at 30 fps:
gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=1920, height=1080, framerate=30/1 ! encodebin profile="video/x-h264" ! qtmux ! filesink location=v_h264_1920x1080_30fps_vbr.mp4
ls -al v_h264_1920x1080_30fps_vbr.mp4 -rw-r--r-- 1 root root 38301010 May 30 23:39 v_h264_1920x1080_30fps_vbr.mp4
Meaning 30 Mb/s for H264 HD at 30 fps.
4.5.2.2. Constant bitrate[edit | edit source]
The encoder can be configured in constant bitrate mode (CBR) to respect a given bitrate. This is useful when dealing with stream transmission while respecting a given bandwidth like a video call for example. This is done by setting the encoder properties rate-control and bitrate.
Here is an example with H264 HD at 30 fps targeting 10 Mb/s:
gst-launch-1.0 -e videotestsrc num-buffers=1000 horizontal-speed=3 ! video/x-raw, width=1920, height=1080, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,bitrate=10000000" ! qtmux ! filesink location=v_h264_1920x1080_30fps_cbr_10Mb.mp4
ls -al v_h264_1920x1080_30fps_cbr_10Mb.mp4 -rw-r--r-- 1 root root 49785575 Jun 5 08:03 v_h264_1920x1080_30fps_cbr_10Mb.mp4
Meaning 11.9 Mb/s for H264 HD at 30 fps. The bitrate is higher than 10 Mb/s at the beginning but then quickly converges to 10 Mb/s.
4.5.3. Controlling encoder keyframe interval[edit | edit source]
Key frame refresh rate can be tuned thanks to encoder property keyframe-interval.
Here is an example to generate one key frame every 30 frames:
gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8|element-properties,keyframe-interval=30" ! matroskamux ! filesink location=v_vp8_640x480_30fps_key_1s.webm
gst-launch-1.0 filesrc location=v_vp8_640x480_30fps_key_1s.webm ! matroskademux ! identity silent=false ! fakesink -v 2>&1 | grep chain | grep -v delta-unit
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (47070 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04040 discont tag-memory , meta: none) 0x217186c0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (19478 bytes, dts: none, pts: 0:00:01.033000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217187e0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11813 bytes, dts: none, pts: 0:00:02.067000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x21718a20 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11688 bytes, dts: none, pts: 0:00:03.100000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217186c0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11627 bytes, dts: none, pts: 0:00:04.133000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217185a0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11576 bytes, dts: none, pts: 0:00:05.167000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x21718a20 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11602 bytes, dts: none, pts: 0:00:06.200000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217186c0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11785 bytes, dts: none, pts: 0:00:07.233000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217185a0 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11515 bytes, dts: none, pts: 0:00:08.267000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x21718a20 /GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain ******* (identity0:sink) (11637 bytes, dts: none, pts: 0:00:09.300000000, duration: 0:00:00.033333333, offset: -1, offset_end: -1, flags: 000 04000 tag-memory , meta: none) 0x217187e0
Ten key frames for 10 s video, which conforms to one key frame every 30 frames at 30 fps.
4.5.4. Controlling encoder rotation[edit | edit source]
Encoder input frame can be rotated by 90 and 270° right before encoding. This is done thanks to the encoder property rotation.
Here is an example with VP8:
gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8|element-properties,rotation=90" ! matroskamux ! filesink location=v_vp8_640x480_30fps_rot90.webm
Here is an example with H264:
gst-launch-1.0 -e videotestsrc num-buffers=300 horizontal-speed=3 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rotation=90" ! qtmux ! filesink location=v_h264_640x480_30fps_rot90.mp4
4.6. Audio/video encoding[edit | edit source]
encodebin can handle both video and audio track by connecting an audio source to audio_0 encodebin pad while specifying a profile that handles both audio and video.
Here is an example with a webm profile that will encode the video to the VP8 format and the audio to the Vorbis format:
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin name=encoder profile="webm" ! filesink location=av_vp8_640x480_vorbis.webm audiotestsrc num-buffers=400 ! encoder.audio_0
Use gst-discoverer to check back encoded content:
gst-discoverer-1.0 av_vp8_640x480_vorbis.webm [...] container #0: WebM video #1: VP8 Width: 640 Height: 480 Depth: 24 Frame rate: 30/1 audio #2: Vorbis Channels: 1 (unknown layout) Sample rate: 44100 Depth: 32 Bitrate: 80000
Here is another example with the Quicktime profile that will encode the video to the H264 format and the audio to the AAC format:
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin name=encoder profile="mp4" ! filesink location=av_h264_640x480_aac.mp4 audiotestsrc num-buffers=400 ! encoder.audio_0
gst-discoverer-1.0 av_h264_640x480_aac.mp4 [...] container #0: Quicktime video #1: H.264 (Main Profile) Width: 640 Height: 480 Depth: 24 Frame rate: 30/1 Pixel aspect ratio: 1/1 Interlaced: false Bitrate: 0 Max bitrate: 0 audio #2: MPEG-4 AAC Channels: 2 (front-left, front-right) Sample rate: 48000 Depth: 32 Bitrate: 0 Max bitrate: 0
4.7. Pictures streaming over network[edit | edit source]
Refer to the How to stream RAW camera over network article to get some examples on how to stream camera content over network using encodebin.
4.8. Audio/video transcoding[edit | edit source]
Refer to the Gst-transcoder article for examples on how to transcode audio/video multimedia content.
4.9. Multi-instance decoding/encoding[edit | edit source]
Several decodings and encodings can be done in parallel without restrictions on the number of instances but respecting the HD at 60 fps maximum performance, for example:
- 2x HD at 30 fps decodings
- 2x HD at 30 fps encodings
- 1x HD at 30 fps decoding and 1x HD at 30 fps encoding
- 6x VGA at 30 fps decodings
- ...
Here is an example of nine video encodings in parallel with nine video decodings. It demonstrates a virtual display wall example with a 480x360 video test pattern divided into 3x3 sub-blocks of 160x120 pixels, each encoded in H264 and sent on local network via UDP. The 3x3 sub-blocks are then decoded and displayed with a small outline between them:
#!/bin/bash # 3x3_h264_wall.sh WIDTH=160 HEIGHT=120 PORT=5000 ADDR=127.0.0.1 gst-launch-1.0 videotestsrc ! video/x-raw, width=$((WIDTH*3)), height=$((HEIGHT*3)) ! queue ! tee name=tee \ tee. ! queue ! videocrop left=$((WIDTH*0)) right=$((WIDTH*2)) top=$((HEIGHT*0)) bottom=$((HEIGHT*2)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+0)) \ tee. ! queue ! videocrop left=$((WIDTH*1)) right=$((WIDTH*1)) top=$((HEIGHT*0)) bottom=$((HEIGHT*2)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+1)) \ tee. ! queue ! videocrop left=$((WIDTH*2)) right=$((WIDTH*0)) top=$((HEIGHT*0)) bottom=$((HEIGHT*2)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+2)) \ tee. ! queue ! videocrop left=$((WIDTH*0)) right=$((WIDTH*2)) top=$((HEIGHT*1)) bottom=$((HEIGHT*1)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+3)) \ tee. ! queue ! videocrop left=$((WIDTH*1)) right=$((WIDTH*1)) top=$((HEIGHT*1)) bottom=$((HEIGHT*1)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+4)) \ tee. ! queue ! videocrop left=$((WIDTH*2)) right=$((WIDTH*0)) top=$((HEIGHT*1)) bottom=$((HEIGHT*1)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+5)) \ tee. ! queue ! videocrop left=$((WIDTH*0)) right=$((WIDTH*2)) top=$((HEIGHT*2)) bottom=$((HEIGHT*0)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+6)) \ tee. ! queue ! videocrop left=$((WIDTH*1)) right=$((WIDTH*1)) top=$((HEIGHT*2)) bottom=$((HEIGHT*0)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+7)) \ tee. ! queue ! videocrop left=$((WIDTH*2)) right=$((WIDTH*0)) top=$((HEIGHT*2)) bottom=$((HEIGHT*0)) ! queue ! v4l2slh264enc ! h264parse config-interval=1 disable-passthrough=1 ! queue ! rtph264pay ! udpsink host=$ADDR port=$((PORT+8)) \ & sleep 1; #+5 to see that 3x3 different tiles WIDTH=$((WIDTH+5)) HEIGHT=$((HEIGHT+5)) gst-launch-1.0 videotestsrc pattern=2 ! video/x-raw, width=$((WIDTH*3)), height=$((HEIGHT*3)) ! queue ! compositor name=mix \ sink_1::xpos=$((WIDTH*0)) sink_1::ypos=$((HEIGHT*0)) \ sink_2::xpos=$((WIDTH*1)) sink_2::ypos=$((HEIGHT*0)) \ sink_3::xpos=$((WIDTH*2)) sink_3::ypos=$((HEIGHT*0)) \ sink_4::xpos=$((WIDTH*0)) sink_4::ypos=$((HEIGHT*1)) \ sink_5::xpos=$((WIDTH*1)) sink_5::ypos=$((HEIGHT*1)) \ sink_6::xpos=$((WIDTH*2)) sink_6::ypos=$((HEIGHT*1)) \ sink_7::xpos=$((WIDTH*0)) sink_7::ypos=$((HEIGHT*2)) \ sink_8::xpos=$((WIDTH*1)) sink_8::ypos=$((HEIGHT*2)) \ sink_9::xpos=$((WIDTH*2)) sink_9::ypos=$((HEIGHT*2)) \ ! queue ! waylandsink sync=false fullscreen=true \ udpsrc port=$((PORT+0)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+1)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+2)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+3)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+4)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+5)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+6)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+7)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ udpsrc port=$((PORT+8)) ! 'application/x-rtp,payload=96,encoding-name=H264,clock-rate=(int)90000' ! rtph264depay ! queue ! h264parse ! v4l2slh264dec ! queue ! mix. \ & sleep 10 killall gst-launch-1.0
chmod +x 3x3_h264_wall.sh ./3x3_h264_wall.sh
5. How to trace and debug[edit | edit source]
5.1. How to monitor[edit | edit source]
5.1.1. Check of devicetree configuration[edit | edit source]
Here are some commands to verify that VDEC and VENC are enabled and log details about the devicetree configuration:
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh
echo "#!/bin/bash" > dtdump.sh;echo "find \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh
rm ~/devicetree.txt echo "[devicetree]" >> ~/devicetree.txt echo "|-[vdec]" >> ~/devicetree.txt dtdump.sh /proc/device-tree/soc/rifsc@42080000/vdec@480d0000 | sed 's/\/proc\/device-tree\/soc\/rifsc@42080000\//| |-/' >> ~/devicetree.txt echo "|" >> ~/devicetree.txt echo "|-[venc]" >> ~/devicetree.txt dtdump.sh /proc/device-tree/soc/rifsc@42080000/venc@480e0000 | sed 's/\/proc\/device-tree\/soc\/rifsc@42080000\//| |-/' >> ~/devicetree.txt echo "|" >> ~/devicetree.txt cat ~/devicetree.txt
[devicetree] |-[vdec] | |-vdec@480d0000/clock-names=vdec-clk 766465632d636c6b00 | |-vdec@480d0000/resets=C 0000000700004320 | |-vdec@480d0000/interrupts=u 000000000000007500000004 | |-vdec@480d0000/clocks=� 00000007000000e5 | |-vdec@480d0000/feature-domains=Y 0000000a00000059 | |-vdec@480d0000/compatible=st,stm32mp25-vdec 73742c73746d33326d7032352d7664656300 | |-vdec@480d0000/interrupt-names=vdec 7664656300 | |-vdec@480d0000/phandle=� 000000db | |-vdec@480d0000/name=vdec 7664656300 | |-[venc] | |-venc@480e0000/clock-names=venc-clk 76656e632d636c6b00 | |-venc@480e0000/resets=C@ 0000000700004340 | |-venc@480e0000/interrupts=性 00000000000000a700000004 | |-venc@480e0000/clocks=� 00000007000000e6 | |-venc@480e0000/feature-domains=Z 0000000a0000005a | |-venc@480e0000/compatible=st,stm32mp25-venc 73742c73746d33326d7032352d76656e6300 | |-venc@480e0000/interrupt-names=venc 76656e6300 | |-venc@480e0000/reg=H 480e000000000800 | |-venc@480e0000/phandle=� 000000dc | |-venc@480e0000/reset-names=venc-rst 76656e632d72737400 | |-venc@480e0000/name=venc 76656e6300
5.1.2. Check interrupts activity[edit | edit source]
To verify that VDEC and/or VENC hardware accelerator is well involved when decoding and/or encoding, one can check the number of interrupts. One interrupt is raised at each frame decoding/encoding.
gst-play-1.0 v_vp8_640x480_30fps.webm cat /proc/interrupts | grep vdec 86: 300 0 GIC-0 149 Level 480d0000.vdec
300 interrupts for 10 s of video at 30 fps.
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! matroskamux ! filesink location=v_vp8_640x480_30fps.webm cat /proc/interrupts | grep venc 87: 300 0 GIC-0 199 Level 480e0000.venc
300 interrupts for 300 frames encoded.
5.2. How to trace[edit | edit source]
5.2.1. V4L2 userland API tracing[edit | edit source]
Tracing of V4L2 userland API[2] can be enabled using command:
echo 0x3 > /sys/devices/platform/soc/42080000.rifsc/480d0000.vdec/video4linux/video0/dev_debug echo 0x3 > /sys/devices/platform/soc/42080000.rifsc/480e0000.venc/video4linux/video1/dev_debug
Traces are output in the kernel log buffer (dmesg).
Trace after a video decoding:
gst-play-1.0 v_vp8_640x480_short.webm dmesg
[ 7540.785769] video0: VIDIOC_QUERYCAP: driver=hantro-vpu, card=st,stm32mp25-vdec-dec, bus=platform: hantro-vpu, version=0x0006011c, capabilities=0x84204000, device_caps=0x04204000 [ 7540.785813] video0: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x1, pixelformat=VP8F little-endian (0x46385056), mbus_code=0x0000, description='VP8 Frame' [ 7540.785858] video0: VIDIOC_S_FMT: type=vid-out-mplane, width=320, height=240, format=VP8F little-endian (0x46385056), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.785883] plane 0: bytesperline=0 sizeimage=262144 [ 7540.785894] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=320, height=240, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.785916] plane 0: bytesperline=320 sizeimage=115200 [ 7540.785951] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.785976] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 7540.786002] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.786039] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 7540.786308] video0: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x1, pixelformat=S264 little-endian (0x34363253), mbus_code=0x0000, description='H.264 Parsed Slice Data' [ 7540.786345] video0: VIDIOC_S_FMT: type=vid-out-mplane, width=320, height=240, format=S264 little-endian (0x34363253), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.786369] plane 0: bytesperline=0 sizeimage=262144 [ 7540.786378] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=320, height=240, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.786401] plane 0: bytesperline=320 sizeimage=134432 [ 7540.786416] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.786440] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 7540.786465] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.786492] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 7540.786519] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40902, type=512, name=H264 Sequence Parameter Set, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=1048, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 7540.786545] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40903, type=513, name=H264 Picture Parameter Set, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=12, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 7540.786571] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40904, type=514, name=H264 Scaling Matrix, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=480, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 7540.786596] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40907, type=516, name=H264 Decode Parameters, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=560, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 7540.786621] video0: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xa40906, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0 [ 7540.786648] video0: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xa40905, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0 [ 7540.786720] video0: VIDIOC_ENUM_FMT: error -22: index=2, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 7540.787879] video0: VIDIOC_QUERYCAP: driver=hantro-vpu, card=st,stm32mp25-vdec-dec, bus=platform: hantro-vpu, version=0x0006011c, capabilities=0x84204000, device_caps=0x04204000 [ 7540.850887] video0: VIDIOC_S_FMT: type=vid-out-mplane, width=640, height=480, format=VP8F little-endian (0x46385056), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.850937] plane 0: bytesperline=0 sizeimage=460800 [ 7540.850964] video0: VIDIOC_S_EXT_CTRLS: which=0x0, count=1, error_idx=0, request_fd=0, name=VP8 Frame Parameters, id/size=0xa409c8/1232 [ 7540.850996] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=640, height=480, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.851019] plane 0: bytesperline=640 sizeimage=460800 [ 7540.851041] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.851069] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 7540.851095] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 7540.851125] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 7540.861871] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=640, height=480, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 7540.861917] plane 0: bytesperline=640 sizeimage=460800 [ 7540.902656] video0: VIDIOC_REQBUFS: count=2, type=vid-out-mplane, memory=mmap [ 7540.902710] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004000, field=any, sequence=0, memory=mmap [ 7540.902734] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 7540.902742] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.902788] video0: VIDIOC_EXPBUF: fd=14, type=vid-out-mplane, index=0, plane=0, flags=0x00080002 [ 7540.902834] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.902856] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 7540.902863] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.902891] video0: VIDIOC_EXPBUF: fd=15, type=vid-out-mplane, index=1, plane=0, flags=0x00080002 [ 7540.903739] video0: VIDIOC_REQBUFS: count=6, type=vid-cap-mplane, memory=mmap [ 7540.903764] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004000, field=any, sequence=0, memory=mmap [ 7540.903786] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.903793] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.903816] video0: VIDIOC_EXPBUF: fd=16, type=vid-cap-mplane, index=0, plane=0, flags=0x00080002 [ 7540.903845] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.903867] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.903874] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.903894] video0: VIDIOC_EXPBUF: fd=17, type=vid-cap-mplane, index=1, plane=0, flags=0x00080002 [ 7540.903947] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.903969] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.903976] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.903998] video0: VIDIOC_EXPBUF: fd=18, type=vid-cap-mplane, index=2, plane=0, flags=0x00080002 [ 7540.904023] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.904044] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.904051] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.904074] video0: VIDIOC_EXPBUF: fd=19, type=vid-cap-mplane, index=3, plane=0, flags=0x00080002 [ 7540.904100] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.904122] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.904129] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.904149] video0: VIDIOC_EXPBUF: fd=20, type=vid-cap-mplane, index=4, plane=0, flags=0x00080002 [ 7540.904174] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 7540.904195] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 7540.904202] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 7540.904241] video0: VIDIOC_EXPBUF: fd=21, type=vid-cap-mplane, index=5, plane=0, flags=0x00080002 [ 7540.904897] video0: VIDIOC_STREAMON: type=vid-out-mplane [ 7540.904922] video0: VIDIOC_STREAMON: type=vid-cap-mplane [ 7540.905172] video0: VIDIOC_S_EXT_CTRLS: which=0xf010000, count=1, error_idx=0, request_fd=26, name=VP8 Frame Parameters, id/size=0xa409c8/1232 [ 7540.905697] video0: VIDIOC_S_EXT_CTRLS: which=0xf010000, count=1, error_idx=0, request_fd=29, name=VP8 Frame Parameters, id/size=0xa409c8/1232 [ 7540.997351] video0: VIDIOC_STREAMOFF: type=vid-out-mplane [ 7540.997457] video0: VIDIOC_STREAMOFF: type=vid-cap-mplane [ 7540.997484] video0: VIDIOC_REQBUFS: count=0, type=vid-out-mplane, memory=mmap [ 7540.997790] video0: VIDIOC_REQBUFS: count=0, type=vid-cap-mplane, memory=mmap
Trace after a video encoding:
gst-launch-1.0 videotestsrc num-buffers=2 ! video/x-raw, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! matroskamux ! filesink location=v_vp8_640x480_short.webm dmesg
[ 8309.136707] video0: VIDIOC_QUERYCAP: driver=hantro-vpu, card=st,stm32mp25-vdec-dec, bus=platform: hantro-vpu, version=0x0006011c, capabilities=0x84204000, device_caps=0x04204000 [ 8309.136751] video0: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x1, pixelformat=VP8F little-endian (0x46385056), mbus_code=0x0000, description='VP8 Frame' [ 8309.136791] video0: VIDIOC_S_FMT: type=vid-out-mplane, width=320, height=240, format=VP8F little-endian (0x46385056), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.136816] plane 0: bytesperline=0 sizeimage=262144 [ 8309.136827] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=320, height=240, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.136849] plane 0: bytesperline=320 sizeimage=115200 [ 8309.136941] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 8309.136967] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.136993] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 8309.137034] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.137271] video0: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x1, pixelformat=S264 little-endian (0x34363253), mbus_code=0x0000, description='H.264 Parsed Slice Data' [ 8309.137304] video0: VIDIOC_S_FMT: type=vid-out-mplane, width=320, height=240, format=S264 little-endian (0x34363253), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.137328] plane 0: bytesperline=0 sizeimage=262144 [ 8309.137337] video0: VIDIOC_G_FMT: type=vid-cap-mplane, width=320, height=240, format=NV12 little-endian (0x3231564e), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.137360] plane 0: bytesperline=320 sizeimage=134432 [ 8309.137375] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 8309.137398] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.137423] video0: VIDIOC_ENUM_FRAMESIZES: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), type=0 [ 8309.137450] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.137476] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40902, type=512, name=H264 Sequence Parameter Set, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=1048, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.137502] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40903, type=513, name=H264 Picture Parameter Set, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=12, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.137527] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40904, type=514, name=H264 Scaling Matrix, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=480, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.137552] video0: VIDIOC_QUERY_EXT_CTRL: id=0xa40907, type=516, name=H264 Decode Parameters, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=560, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.137577] video0: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xa40906, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0 [ 8309.137603] video0: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xa40905, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0 [ 8309.137672] video0: VIDIOC_ENUM_FMT: error -22: index=2, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.137920] video1: VIDIOC_QUERYCAP: driver=hantro-vpu, card=st,stm32mp25-venc-enc, bus=platform: hantro-vpu, version=0x0006011c, capabilities=0x84204000, device_caps=0x04204000 [ 8309.137946] video1: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x1, pixelformat=JPEG little-endian (0x4745504a), mbus_code=0x0000, description='JFIF JPEG' [ 8309.137968] video1: VIDIOC_ENUM_FMT: index=1, type=vid-cap-mplane, flags=0x1, pixelformat=VP8F little-endian (0x46385056), mbus_code=0x0000, description='VP8 Frame' [ 8309.137995] video1: VIDIOC_QUERY_EXT_CTRL: id=0xa40af5, type=640, name=VP8 Encode Parameters, min/max=0/0, step=0, default=0, flags=0x00000100, elem_size=20, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.138021] video1: VIDIOC_QUERY_EXT_CTRL: id=0xa40af6, type=1, name=VP8 Encode QP, min/max=0/127, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.138137] video1: VIDIOC_ENUM_FMT: error -22: index=2, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.168457] video1: VIDIOC_QUERYCAP: driver=hantro-vpu, card=st,stm32mp25-venc-enc, bus=platform: hantro-vpu, version=0x0006011c, capabilities=0x84204000, device_caps=0x04204000 [ 8309.173895] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.173937] plane 0: bytesperline=96 sizeimage=4608 [ 8309.173968] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.173994] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.174017] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.174040] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.174062] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.174084] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.174106] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.174130] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.174153] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.174176] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.174200] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.174223] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.174245] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.186999] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.187049] plane 0: bytesperline=96 sizeimage=4608 [ 8309.187077] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.187103] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.187126] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.187149] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.187172] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.187193] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.187215] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.187238] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.187261] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.187284] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.187309] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.187333] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.187356] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.205616] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.205669] plane 0: bytesperline=96 sizeimage=4608 [ 8309.205701] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.205728] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.205752] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.205775] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.205798] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.205820] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.205842] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.205866] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.205889] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.205911] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.205936] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.205959] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.205981] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.216242] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.216287] plane 0: bytesperline=96 sizeimage=4608 [ 8309.216315] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.216341] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.216364] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.216387] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.216411] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.216432] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.216455] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.216479] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.216502] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.216525] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.216550] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.216573] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.216596] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.222522] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.222565] plane 0: bytesperline=96 sizeimage=4608 [ 8309.222592] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.222620] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.222644] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.222667] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.222690] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.222711] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.222734] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.222759] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.222783] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.222805] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.222830] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.222854] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.222877] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.230210] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.230253] plane 0: bytesperline=96 sizeimage=4608 [ 8309.230279] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.230306] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.230330] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.230352] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.230375] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.230397] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.230420] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.230443] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.230467] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.230490] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.230515] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.230538] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.230561] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.231251] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.231279] plane 0: bytesperline=96 sizeimage=4608 [ 8309.231296] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.231321] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.231344] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.231367] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.231389] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.231411] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.231433] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.231456] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.231478] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.231500] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.231525] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.231549] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.231572] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.231938] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=96, height=32, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.231968] plane 0: bytesperline=96 sizeimage=4608 [ 8309.231986] video1: VIDIOC_ENUM_FMT: index=0, type=vid-out-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0' [ 8309.232011] video1: VIDIOC_ENUM_FMT: index=1, type=vid-out-mplane, flags=0x0, pixelformat=YM12 little-endian (0x32314d59), mbus_code=0x0000, description='Planar YUV 4:2:0 (N-C)' [ 8309.232034] video1: VIDIOC_ENUM_FMT: index=2, type=vid-out-mplane, flags=0x0, pixelformat=NM12 little-endian (0x32314d4e), mbus_code=0x0000, description='Y/UV 4:2:0 (N-C)' [ 8309.232057] video1: VIDIOC_ENUM_FMT: index=3, type=vid-out-mplane, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2' [ 8309.232080] video1: VIDIOC_ENUM_FMT: index=4, type=vid-out-mplane, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2' [ 8309.232101] video1: VIDIOC_ENUM_FMT: index=5, type=vid-out-mplane, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5' [ 8309.232123] video1: VIDIOC_ENUM_FMT: index=6, type=vid-out-mplane, flags=0x0, pixelformat=XR24 little-endian (0x34325258), mbus_code=0x0000, description='32-bit BGRX 8-8-8-8' [ 8309.232147] video1: VIDIOC_ENUM_FMT: index=7, type=vid-out-mplane, flags=0x0, pixelformat=BGR4 little-endian (0x34524742), mbus_code=0x0000, description='32-bit BGRA/X 8-8-8-8' [ 8309.232170] video1: VIDIOC_ENUM_FMT: index=8, type=vid-out-mplane, flags=0x0, pixelformat=XB24 little-endian (0x34324258), mbus_code=0x0000, description='32-bit RGBX 8-8-8-8' [ 8309.232193] video1: VIDIOC_ENUM_FMT: index=9, type=vid-out-mplane, flags=0x0, pixelformat=RX24 little-endian (0x34325852), mbus_code=0x0000, description='32-bit XBGR 8-8-8-8' [ 8309.232218] video1: VIDIOC_ENUM_FMT: index=10, type=vid-out-mplane, flags=0x0, pixelformat=BX24 little-endian (0x34325842), mbus_code=0x0000, description='32-bit XRGB 8-8-8-8' [ 8309.232241] video1: VIDIOC_ENUM_FMT: index=11, type=vid-out-mplane, flags=0x0, pixelformat=RGB4 little-endian (0x34424752), mbus_code=0x0000, description='32-bit A/XRGB 8-8-8-8' [ 8309.232263] video1: VIDIOC_ENUM_FMT: error -22: index=12, type=vid-out-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description='' [ 8309.232619] video1: VIDIOC_STREAMOFF: type=vid-out-mplane [ 8309.232642] video1: VIDIOC_STREAMOFF: type=vid-cap-mplane [ 8309.232659] video1: VIDIOC_G_FMT: type=vid-cap-mplane, width=96, height=32, format=JPEG little-endian (0x4745504a), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0 [ 8309.232684] plane 0: bytesperline=0 sizeimage=6768 [ 8309.232701] video1: VIDIOC_S_FMT: type=vid-cap-mplane, width=640, height=480, format=VP8F little-endian (0x46385056), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.232725] plane 0: bytesperline=0 sizeimage=460800 [ 8309.232734] video1: VIDIOC_G_FMT: type=vid-out-mplane, width=640, height=480, format=NV12 little-endian (0x3231564e), field=none, colorspace=7, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_fun0 [ 8309.232757] plane 0: bytesperline=640 sizeimage=460800 [ 8309.240286] video1: VIDIOC_REQBUFS: count=4, type=vid-out-mplane, memory=mmap [ 8309.240353] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004000, field=any, sequence=0, memory=mmap [ 8309.240377] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 8309.240385] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.240441] video1: VIDIOC_EXPBUF: fd=12, type=vid-out-mplane, index=0, plane=0, flags=0x00080002 [ 8309.240525] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.240548] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 8309.240555] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.240580] video1: VIDIOC_EXPBUF: fd=13, type=vid-out-mplane, index=1, plane=0, flags=0x00080002 [ 8309.240609] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.240632] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 8309.240639] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.240661] video1: VIDIOC_EXPBUF: fd=14, type=vid-out-mplane, index=2, plane=0, flags=0x00080002 [ 8309.240689] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-out-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.240711] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800 [ 8309.240718] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.240740] video1: VIDIOC_EXPBUF: fd=15, type=vid-out-mplane, index=3, plane=0, flags=0x00080002 [ 8309.248249] video1: VIDIOC_REQBUFS: count=4, type=vid-cap-mplane, memory=mmap [ 8309.248315] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004000, field=any, sequence=0, memory=mmap [ 8309.248338] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 8309.248347] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.248398] video1: VIDIOC_EXPBUF: fd=18, type=vid-cap-mplane, index=0, plane=0, flags=0x00080002 [ 8309.248452] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.248475] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 8309.248482] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.248507] video1: VIDIOC_EXPBUF: fd=19, type=vid-cap-mplane, index=1, plane=0, flags=0x00080002 [ 8309.248534] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.248556] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 8309.248563] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.248585] video1: VIDIOC_EXPBUF: fd=20, type=vid-cap-mplane, index=2, plane=0, flags=0x00080002 [ 8309.248611] video1: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap-mplane, request_fd=0, flags=0x00004001, field=any, sequence=0, memory=mmap [ 8309.248633] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x40000000, length=460800 [ 8309.248640] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 8309.248662] video1: VIDIOC_EXPBUF: fd=21, type=vid-cap-mplane, index=3, plane=0, flags=0x00080002 [ 8309.249682] video1: VIDIOC_STREAMON: type=vid-out-mplane [ 8309.251027] video1: VIDIOC_STREAMON: type=vid-cap-mplane [ 8309.251058] video1: VIDIOC_QUERY_EXT_CTRL: id=0xa40af6, type=1, name=VP8 Encode QP, min/max=0/127, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0 [ 8309.252133] video1: VIDIOC_S_EXT_CTRLS: which=0xf010000, count=2, error_idx=1, request_fd=24, name=VP8 Encode Parameters, id/size=0xa40af5/20, name=VP8 Encode QP, id/size=0xa40af6/4 [ 8309.257729] video1: VIDIOC_S_EXT_CTRLS: which=0xf010000, count=2, error_idx=1, request_fd=24, name=VP8 Encode Parameters, id/size=0xa40af5/20, name=VP8 Encode QP, id/size=0xa40af6/4 [ 8309.263950] video1: VIDIOC_STREAMOFF: type=vid-out-mplane [ 8309.264171] video1: VIDIOC_STREAMOFF: type=vid-cap-mplane [ 8309.264207] video1: VIDIOC_REQBUFS: count=0, type=vid-out-mplane, memory=mmap [ 8309.264647] video1: VIDIOC_REQBUFS: count=0, type=vid-cap-mplane, memory=mmap
5.2.2. V4L2 core framework tracing[edit | edit source]
Tracing of the V4L2 core framework[4] can be enabled using commands:
echo 0x3 > /sys/module/videobuf2_common/parameters/debug echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug
Traces are output in the kernel log buffer (dmesg).
Trace after a video decoding:
dmesg
[ 8729.602252] videobuf2_common: [out-00000000809b1516] __setup_offsets: buffer 0, plane 0 offset 0x00000000 [ 8729.611459] videobuf2_common: [out-00000000809b1516] __setup_offsets: buffer 1, plane 0 offset 0x00071000 [ 8729.619976] videobuf2_common: [out-00000000809b1516] __vb2_queue_alloc: allocated 2 buffers, 1 plane(s) each [ 8729.629864] videobuf2_common: [out-00000000809b1516] vb2_core_expbuf: buffer 0, plane 0 exported as 14 descriptor [ 8729.640129] videobuf2_common: [out-00000000809b1516] vb2_core_expbuf: buffer 1, plane 0 exported as 15 descriptor [ 8729.652433] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 0, plane 0 offset 0x00000000 [ 8729.662330] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 1, plane 0 offset 0x00071000 [ 8729.671509] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 2, plane 0 offset 0x000e2000 [ 8729.681304] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 3, plane 0 offset 0x00153000 [ 8729.690711] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 4, plane 0 offset 0x001c4000 [ 8729.700356] videobuf2_common: [cap-00000000942614f9] __setup_offsets: buffer 5, plane 0 offset 0x00235000 [ 8729.708688] videobuf2_common: [cap-00000000942614f9] __vb2_queue_alloc: allocated 6 buffers, 1 plane(s) each [ 8729.718568] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 0, plane 0 exported as 16 descriptor [ 8729.728844] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 1, plane 0 exported as 17 descriptor [ 8729.739135] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 2, plane 0 exported as 18 descriptor [ 8729.749300] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 3, plane 0 exported as 19 descriptor [ 8729.759544] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 4, plane 0 exported as 20 descriptor [ 8729.769849] videobuf2_common: [cap-00000000942614f9] vb2_core_expbuf: buffer 5, plane 0 exported as 21 descriptor [ 8729.780703] videobuf2_common: [out-00000000809b1516] vb2_core_streamon: successful [ 8729.787519] videobuf2_common: [cap-00000000942614f9] vb2_core_streamon: successful [ 8729.795478] videobuf2_common: [out-00000000809b1516] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8729.803682] videobuf2_common: [cap-00000000942614f9] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8729.812289] videobuf2_common: [out-00000000809b1516] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8729.821400] videobuf2_common: [out-00000000809b1516] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8729.829507] videobuf2_common: [cap-00000000942614f9] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8729.838000] videobuf2_common: [out-00000000809b1516] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8729.846664] videobuf2_common: [out-00000000809b1516] vb2_core_dqbuf: returning done buffer [ 8729.854853] videobuf2_common: [out-00000000809b1516] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued [ 8729.864264] videobuf2_common: [cap-00000000942614f9] vb2_core_dqbuf: returning done buffer [ 8729.872548] videobuf2_common: [cap-00000000942614f9] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued [ 8729.892426] videobuf2_common: [out-00000000809b1516] vb2_core_dqbuf: returning done buffer [ 8729.903680] videobuf2_common: [out-00000000809b1516] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued [ 8729.915020] videobuf2_common: [cap-00000000942614f9] vb2_core_dqbuf: returning done buffer [ 8729.922151] videobuf2_common: [cap-00000000942614f9] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued [ 8729.972009] videobuf2_common: [out-00000000809b1516] vb2_core_streamoff: successful [ 8729.979418] videobuf2_common: [cap-00000000942614f9] vb2_core_streamoff: successful [ 8729.987113] videobuf2_common: [out-00000000809b1516] vb2_core_reqbufs: memory in use, orphaning buffers [ 8729.996517] videobuf2_common: [out-00000000809b1516] __vb2_buf_mem_free: freed plane 0 of buffer 0 [ 8730.005414] videobuf2_common: [out-00000000809b1516] __vb2_buf_mem_free: freed plane 0 of buffer 1 [ 8730.014664] videobuf2_common: [cap-00000000942614f9] vb2_core_reqbufs: memory in use, orphaning buffers [ 8730.023759] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 0 [ 8730.032649] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 1 [ 8730.041657] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 2 [ 8730.050510] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 3 [ 8730.059522] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 4 [ 8730.068421] videobuf2_common: [cap-00000000942614f9] __vb2_buf_mem_free: freed plane 0 of buffer 5
Trace after a video encoding:
dmesg
[ 8578.924729] videobuf2_common: [out-000000006798f690] vb2_core_streamoff: successful [ 8578.931032] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_streamoff: successful [ 8578.941950] videobuf2_common: [out-000000006798f690] __setup_offsets: buffer 0, plane 0 offset 0x00000000 [ 8578.951686] videobuf2_common: [out-000000006798f690] __setup_offsets: buffer 1, plane 0 offset 0x00071000 [ 8578.961691] videobuf2_common: [out-000000006798f690] __setup_offsets: buffer 2, plane 0 offset 0x000e2000 [ 8578.971585] videobuf2_common: [out-000000006798f690] __setup_offsets: buffer 3, plane 0 offset 0x00153000 [ 8578.979824] videobuf2_common: [out-000000006798f690] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each [ 8578.990407] videobuf2_common: [out-000000006798f690] vb2_core_expbuf: buffer 0, plane 0 exported as 12 descriptor [ 8578.999972] videobuf2_common: [out-000000006798f690] vb2_core_expbuf: buffer 1, plane 0 exported as 13 descriptor [ 8579.010114] videobuf2_common: [out-000000006798f690] vb2_core_expbuf: buffer 2, plane 0 exported as 14 descriptor [ 8579.020335] videobuf2_common: [out-000000006798f690] vb2_core_expbuf: buffer 3, plane 0 exported as 15 descriptor [ 8579.032594] videobuf2_common: [cap-00000000cbeec6f9] __setup_offsets: buffer 0, plane 0 offset 0x00000000 [ 8579.041845] videobuf2_common: [cap-00000000cbeec6f9] __setup_offsets: buffer 1, plane 0 offset 0x00071000 [ 8579.052201] videobuf2_common: [cap-00000000cbeec6f9] __setup_offsets: buffer 2, plane 0 offset 0x000e2000 [ 8579.062087] videobuf2_common: [cap-00000000cbeec6f9] __setup_offsets: buffer 3, plane 0 offset 0x00153000 [ 8579.070339] videobuf2_common: [cap-00000000cbeec6f9] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each [ 8579.080100] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_expbuf: buffer 0, plane 0 exported as 18 descriptor [ 8579.090375] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_expbuf: buffer 1, plane 0 exported as 19 descriptor [ 8579.100641] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_expbuf: buffer 2, plane 0 exported as 20 descriptor [ 8579.110920] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_expbuf: buffer 3, plane 0 exported as 21 descriptor [ 8579.122030] videobuf2_common: [out-000000006798f690] vb2_core_streamon: successful [ 8579.133472] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_streamon: successful [ 8579.140835] videobuf2_common: [out-000000006798f690] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8579.148292] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8579.157014] videobuf2_common: [out-000000006798f690] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 8579.168099] videobuf2_common: [out-000000006798f690] vb2_core_dqbuf: returning done buffer [ 8579.174975] videobuf2_common: [out-000000006798f690] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued [ 8579.184357] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_dqbuf: returning done buffer [ 8579.192624] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued [ 8579.203489] videobuf2_common: [out-000000006798f690] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8579.212259] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8579.220759] videobuf2_common: [out-000000006798f690] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 8579.232331] videobuf2_common: [out-000000006798f690] vb2_core_dqbuf: returning done buffer [ 8579.240550] videobuf2_common: [out-000000006798f690] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued [ 8579.250421] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_dqbuf: returning done buffer [ 8579.258355] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued [ 8579.271795] videobuf2_common: [out-000000006798f690] vb2_core_streamoff: successful [ 8579.279386] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_streamoff: successful [ 8579.286914] videobuf2_common: [out-000000006798f690] vb2_core_reqbufs: memory in use, orphaning buffers [ 8579.296253] videobuf2_common: [out-000000006798f690] __vb2_buf_mem_free: freed plane 0 of buffer 0 [ 8579.305177] videobuf2_common: [out-000000006798f690] __vb2_buf_mem_free: freed plane 0 of buffer 1 [ 8579.314090] videobuf2_common: [out-000000006798f690] __vb2_buf_mem_free: freed plane 0 of buffer 2 [ 8579.323092] videobuf2_common: [out-000000006798f690] __vb2_buf_mem_free: freed plane 0 of buffer 3 [ 8579.332442] videobuf2_common: [cap-00000000cbeec6f9] vb2_core_reqbufs: memory in use, orphaning buffers [ 8579.341365] videobuf2_common: [cap-00000000cbeec6f9] __vb2_buf_mem_free: freed plane 0 of buffer 0 [ 8579.350251] videobuf2_common: [cap-00000000cbeec6f9] __vb2_buf_mem_free: freed plane 0 of buffer 1 [ 8579.359290] videobuf2_common: [cap-00000000cbeec6f9] __vb2_buf_mem_free: freed plane 0 of buffer 2 [ 8579.368194] videobuf2_common: [cap-00000000cbeec6f9] __vb2_buf_mem_free: freed plane 0 of buffer 3
5.2.3. hantro_vpu V4L2 kernel driver tracing[edit | edit source]
hantro_vpu debug traces can be enabled using the command:
echo <level> > /sys/module/hantro_vpu/parameters/debug
/** * DOC: hantro_debug: Module parameter to control level of debugging messages. * * Level of debugging messages can be controlled by bits of * module parameter called "debug". Meaning of particular * bits is as follows: * * bit 0 - global information: mode, size, init, release * bit 1 - each run start/result information * bit 2 - contents of small controls from userspace * bit 3 - contents of big controls from userspace * bit 4 - detail fmt, ctrl, buffer q/dq information * bit 5 - detail function enter/leave trace information * bit 6 - register write/read information */
Here is an example with a basic video decoding:
echo 0x3f > /sys/module/hantro_vpu/parameters/debug
gst-play-1.0 v_vp8_640x480_short.webm dmesg
[12200.721500] hantro_try_fmt:289: trying format VP8F [12200.724904] hantro_try_fmt:289: trying format NV12 [12200.729691] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.735403] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.740362] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.745516] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12200.750670] hantro_try_fmt:289: trying format NV12 [12200.755420] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.760872] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.766163] hantro_try_fmt:289: trying format VP8F [12200.770830] hantro_try_fmt:289: trying format NV12 [12200.775594] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.781049] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12200.786303] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.791679] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12200.796959] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12200.802316] hantro_try_fmt:289: trying format S264 [12200.806700] hantro_try_fmt:289: trying format NV12 [12200.811456] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.816908] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12200.822163] hantro_set_fmt_out:546: OUTPUT codec mode: 1 [12200.827520] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12200.832793] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12200.838162] hantro_try_fmt:289: trying format JPEG [12200.842584] hantro_try_fmt:289: trying format NV12 [12200.847247] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12200.852700] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12200.857760] hantro_set_fmt_cap:606: CAPTURE codec mode: 0 [12200.863112] hantro_set_fmt_cap:607: fmt - w: 96, h: 32 [12200.868263] hantro_try_fmt:289: trying format NV12 [12200.873021] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12200.878479] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12200.883567] hantro_jpeg_s_ctrl:297: s_ctrl: id = 10291459, val = 50 [12200.891019] hantro_try_fmt:289: trying format VP8F [12200.894636] hantro_try_fmt:289: trying format NV12 [12200.899425] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.904790] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.909950] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.915200] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12200.920359] hantro_try_fmt:289: trying format NV12 [12200.925109] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.930565] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.996426] hantro_try_fmt:289: trying format VP8F [12200.999782] hantro_try_fmt:289: trying format NV12 [12201.004666] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12201.010038] hantro_set_fmt_cap:607: fmt - w: 640, h: 480 [12201.015401] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12201.020668] hantro_set_fmt_out:547: fmt - w: 640, h: 480 [12201.025982] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12201.041725] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12201.090320] hantro_start_streaming:902: Codec mode = 3 [12201.094498] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.101460] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.109134] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.115992] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800
With VDEC hardware registers read/write details (bit 6):
echo 0x5f > /sys/module/hantro_vpu/parameters/debug
[12200.721500] hantro_try_fmt:289: trying format VP8F [12200.724904] hantro_try_fmt:289: trying format NV12 [12200.729691] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.735403] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.740362] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.745516] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12200.750670] hantro_try_fmt:289: trying format NV12 [12200.755420] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.760872] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.766163] hantro_try_fmt:289: trying format VP8F [12200.770830] hantro_try_fmt:289: trying format NV12 [12200.775594] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.781049] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12200.786303] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.791679] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12200.796959] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12200.802316] hantro_try_fmt:289: trying format S264 [12200.806700] hantro_try_fmt:289: trying format NV12 [12200.811456] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.816908] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12200.822163] hantro_set_fmt_out:546: OUTPUT codec mode: 1 [12200.827520] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12200.832793] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12200.838162] hantro_try_fmt:289: trying format JPEG [12200.842584] hantro_try_fmt:289: trying format NV12 [12200.847247] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12200.852700] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12200.857760] hantro_set_fmt_cap:606: CAPTURE codec mode: 0 [12200.863112] hantro_set_fmt_cap:607: fmt - w: 96, h: 32 [12200.868263] hantro_try_fmt:289: trying format NV12 [12200.873021] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12200.878479] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12200.883567] hantro_jpeg_s_ctrl:297: s_ctrl: id = 10291459, val = 50 [12200.891019] hantro_try_fmt:289: trying format VP8F [12200.894636] hantro_try_fmt:289: trying format NV12 [12200.899425] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.904790] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.909950] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12200.915200] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12200.920359] hantro_try_fmt:289: trying format NV12 [12200.925109] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12200.930565] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12200.996426] hantro_try_fmt:289: trying format VP8F [12200.999782] hantro_try_fmt:289: trying format NV12 [12201.004666] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12201.010038] hantro_set_fmt_cap:607: fmt - w: 640, h: 480 [12201.015401] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12201.020668] hantro_set_fmt_out:547: fmt - w: 640, h: 480 [12201.025982] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12201.041725] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12201.090320] hantro_start_streaming:902: Codec mode = 3 [12201.094498] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.101460] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.109134] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12201.115992] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12456.605725] hantro_try_fmt:289: trying format VP8F [12456.609107] hantro_try_fmt:289: trying format NV12 [12456.613862] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.619336] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12456.624494] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12456.629744] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12456.634897] hantro_try_fmt:289: trying format NV12 [12456.639646] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.645350] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12456.650517] hantro_try_fmt:289: trying format VP8F [12456.655216] hantro_try_fmt:289: trying format NV12 [12456.659972] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.665242] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12456.670634] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12456.675886] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12456.681168] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12456.686543] hantro_try_fmt:289: trying format S264 [12456.690924] hantro_try_fmt:289: trying format NV12 [12456.695709] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.701163] hantro_set_fmt_cap:607: fmt - w: 320, h: 240 [12456.706417] hantro_set_fmt_out:546: OUTPUT codec mode: 1 [12456.711669] hantro_set_fmt_out:547: fmt - w: 320, h: 240 [12456.717083] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12456.722344] hantro_try_fmt:289: trying format JPEG [12456.726734] hantro_try_fmt:289: trying format NV12 [12456.731496] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12456.736847] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12456.742004] hantro_set_fmt_cap:606: CAPTURE codec mode: 0 [12456.747388] hantro_set_fmt_cap:607: fmt - w: 96, h: 32 [12456.752550] hantro_try_fmt:289: trying format NV12 [12456.757310] hantro_set_fmt_out:546: OUTPUT codec mode: -1 [12456.762669] hantro_set_fmt_out:547: fmt - w: 96, h: 32 [12456.767860] hantro_jpeg_s_ctrl:297: s_ctrl: id = 10291459, val = 50 [12456.775224] hantro_try_fmt:289: trying format VP8F [12456.778831] hantro_try_fmt:289: trying format NV12 [12456.783597] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.789049] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12456.794201] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12456.799524] hantro_set_fmt_out:547: fmt - w: 96, h: 96 [12456.804592] hantro_try_fmt:289: trying format NV12 [12456.809359] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.814810] hantro_set_fmt_cap:607: fmt - w: 96, h: 96 [12456.880680] hantro_try_fmt:289: trying format VP8F [12456.884065] hantro_try_fmt:289: trying format NV12 [12456.888821] hantro_set_fmt_cap:606: CAPTURE codec mode: -1 [12456.894279] hantro_set_fmt_cap:607: fmt - w: 640, h: 480 [12456.899672] hantro_set_fmt_out:546: OUTPUT codec mode: 3 [12456.904932] hantro_set_fmt_out:547: fmt - w: 640, h: 480 [12456.910293] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12456.925967] vidioc_g_fmt_cap_mplane:272: f->type = 9 [12456.974393] hantro_start_streaming:902: Codec mode = 3 [12456.978634] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12456.985813] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12456.992743] vdpu_write:412: 0x0001 = 0x00000010 [12456.997215] vdpu_write:412: 0x0002 = 0x00000400 [12457.001805] vdpu_write:412: 0x0065 = 0x00000001 [12457.006305] vdpu_write_relaxed:406: 0x0002 = 0xff7c0363 [12457.011474] vdpu_write_relaxed:406: 0x0003 = 0xa00000ff [12457.016639] vdpu_write_relaxed:406: 0x0004 = 0x1400f000 [12457.021895] vdpu_write_relaxed:406: 0x0005 = 0x000065c0 [12457.027074] vdpu_write_relaxed:406: 0x0007 = 0x00003000 [12457.032338] vdpu_read:427: 0x0020 = 0x00300000 [12457.036693] vdpu_write_relaxed:406: 0x0020 = 0x00680000 [12457.041957] vdpu_write_relaxed:406: 0x001e = 0x00000000 [12457.047125] vdpu_read:427: 0x0021 = 0x00000800 [12457.051593] vdpu_write_relaxed:406: 0x0021 = 0x00000000 [12457.056787] vdpu_read:427: 0x0021 = 0x00000000 [12457.061250] vdpu_write_relaxed:406: 0x0021 = 0x00000000 [12457.066405] vdpu_read:427: 0x0021 = 0x00000000 [12457.070858] vdpu_write_relaxed:406: 0x0021 = 0x00000000 [12457.076008] vdpu_read:427: 0x002e = 0x00000000 [12457.080467] vdpu_write_relaxed:406: 0x002e = 0x00000000 [12457.085728] vdpu_read:427: 0x002e = 0x00000000 [12457.090075] vdpu_write_relaxed:406: 0x002e = 0x00000000 [12457.095325] vdpu_read:427: 0x002f = 0x00000000 [12457.099770] vdpu_write_relaxed:406: 0x002f = 0x00000000 [12457.104921] vdpu_write_relaxed:406: 0x001b = 0xf1900010 [12457.110197] vdpu_read:427: 0x0005 = 0x000065c0 [12457.114553] vdpu_write_relaxed:406: 0x0005 = 0x002c65c0 [12457.119810] vdpu_read:427: 0x0009 = 0x00000ce6 [12457.124255] vdpu_write_relaxed:406: 0x0009 = 0x000009f2 [12457.129408] vdpu_read:427: 0x0009 = 0x000009f2 [12457.133854] vdpu_write_relaxed:406: 0x0009 = 0x000009f2 [12457.139106] vdpu_write_relaxed:406: 0x0006 = 0x0000adde [12457.144255] vdpu_read:427: 0x000c = 0xf198ad93 [12457.148711] vdpu_write_relaxed:406: 0x000c = 0xf1900a00 [12457.153873] vdpu_read:427: 0x0005 = 0x002c65c0 [12457.158323] vdpu_write_relaxed:406: 0x0005 = 0x202c65c0 [12457.163579] vdpu_write_relaxed:406: 0x000e = 0xf1a00000 [12457.168733] vdpu_write_relaxed:406: 0x0012 = 0xf1a00000 [12457.173991] vdpu_write_relaxed:406: 0x0013 = 0xf1a00000 [12457.179150] vdpu_write_relaxed:406: 0x0028 = 0x8e45a000 [12457.184445] vdpu_write_relaxed:406: 0x000a = 0x84e71000 [12457.189616] vdpu_write_relaxed:406: 0x000d = 0xf1a00000 [12457.194793] vdpu_write:412: 0x0001 = 0x00000001 [12457.199820] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12457.206605] vdpu_read:427: 0x0001 = 0x00001100 [12457.206613] vdpu_write:412: 0x0001 = 0x00000000 [12457.206619] vdpu_write:412: 0x0002 = 0x00000400 [12457.220041] hantro_buf_plane_check:812: plane 0 size: 460800, sizeimage: 460800 [12457.227337] vdpu_write:412: 0x0001 = 0x00000010 [12457.231933] vdpu_write:412: 0x0002 = 0x00000400 [12457.236417] vdpu_write:412: 0x0065 = 0x00000001 [12457.240924] vdpu_write_relaxed:406: 0x0002 = 0xff7c0363 [12457.246107] vdpu_write_relaxed:406: 0x0003 = 0xa01000ff [12457.251290] vdpu_write_relaxed:406: 0x0004 = 0x1400f000 [12457.256581] vdpu_write_relaxed:406: 0x0005 = 0x000000c8 [12457.261752] vdpu_write_relaxed:406: 0x0007 = 0x00003000 [12457.266921] vdpu_read:427: 0x0020 = 0x00680000 [12457.271373] vdpu_write_relaxed:406: 0x0020 = 0x00300000 [12457.276523] vdpu_write_relaxed:406: 0x001e = 0x00000000 [12457.281780] vdpu_read:427: 0x0021 = 0x00000000 [12457.286236] vdpu_write_relaxed:406: 0x0021 = 0x00000800 [12457.291401] vdpu_read:427: 0x0021 = 0x00000800 [12457.295858] vdpu_write_relaxed:406: 0x0021 = 0x00000800 [12457.301009] vdpu_read:427: 0x0021 = 0x00000800 [12457.305468] vdpu_write_relaxed:406: 0x0021 = 0x00000800 [12457.310619] vdpu_read:427: 0x002e = 0x00000000 [12457.315065] vdpu_write_relaxed:406: 0x002e = 0x00000000 [12457.320320] vdpu_read:427: 0x002e = 0x00000000 [12457.324773] vdpu_write_relaxed:406: 0x002e = 0x00000000 [12457.329940] vdpu_read:427: 0x002f = 0x00000000 [12457.334402] vdpu_write_relaxed:406: 0x002f = 0x00000000 [12457.339562] vdpu_write_relaxed:406: 0x001b = 0xf1980080 [12457.344814] vdpu_read:427: 0x0005 = 0x000000c8 [12457.349159] vdpu_write_relaxed:406: 0x0005 = 0x001400c8 [12457.354409] vdpu_read:427: 0x0009 = 0x000009f2 [12457.358854] vdpu_write_relaxed:406: 0x0009 = 0x00000ce6 [12457.364004] vdpu_read:427: 0x0009 = 0x00000ce6 [12457.368450] vdpu_write_relaxed:406: 0x0009 = 0x00000ce6 [12457.373712] vdpu_write_relaxed:406: 0x0006 = 0x0000a034 [12457.378884] vdpu_read:427: 0x000c = 0xf190b7dd [12457.383335] vdpu_write_relaxed:406: 0x000c = 0xf1980d60 [12457.388491] vdpu_read:427: 0x0005 = 0x001400c8 [12457.392948] vdpu_write_relaxed:406: 0x0005 = 0xa01400c8 [12457.398216] vdpu_write_relaxed:406: 0x000e = 0xf1a00000 [12457.403370] vdpu_write_relaxed:406: 0x0012 = 0xf1a00000 [12457.408621] vdpu_write_relaxed:406: 0x0013 = 0xf1a00000 [12457.417072] vdpu_write_relaxed:406: 0x0028 = 0x8e45a000 [12457.420918] vdpu_write_relaxed:406: 0x000a = 0x84e71000 [12457.426085] vdpu_write_relaxed:406: 0x000d = 0xf1a80000 [12457.431359] vdpu_write:412: 0x0001 = 0x00000001 [12457.437340] vdpu_read:427: 0x0001 = 0x00001100 [12457.440381] vdpu_write:412: 0x0001 = 0x00000000 [12457.444911] vdpu_write:412: 0x0002 = 0x00000400
5.3. How to debug[edit | edit source]
5.3.1. Errors[edit | edit source]
Errors are unconditionally traced in the kernel log.
5.3.2. Memory tracking[edit | edit source]
Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through DMA back-end. Those allocations can be traced using:
echo "module dma_contiguous +p" > /sys/kernel/debug/dynamic_debug/control echo "module videobuf2_dma_contig +p" > /sys/kernel/debug/dynamic_debug/control
Here is the trace after a VGA VP8 video playback.
[ 1016.160954] vb2_dc_mmap: mapped dma addr 0xf1900000 at 0xffff9e3bd000, size 462848 [ 1016.161421] vb2_dc_mmap: mapped dma addr 0xf1980000 at 0xffff9e34c000, size 462848 [ 1016.176991] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xffffa8b82000, size 462848 [ 1016.177092] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xffffa8b11000, size 462848 [ 1016.215467] vb2_dc_mmap: mapped dma addr 0xf1a80000 at 0xffffa8aa0000, size 462848 [ 1016.215552] vb2_dc_mmap: mapped dma addr 0xf1a80000 at 0xffffa8a2f000, size 462848 [ 1016.249726] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xffffa89be000, size 462848 [ 1016.249889] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xffffa894d000, size 462848 [ 1016.282602] vb2_dc_mmap: mapped dma addr 0xf1b80000 at 0xffffa88dc000, size 462848 [ 1016.282754] vb2_dc_mmap: mapped dma addr 0xf1b80000 at 0xffffa886b000, size 462848 [ 1016.315520] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xffffa87fa000, size 462848 [ 1016.315670] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xffffa8789000, size 462848 [ 1016.353368] vb2_dc_mmap: mapped dma addr 0xf1c80000 at 0xffffa8718000, size 462848 [ 1016.353495] vb2_dc_mmap: mapped dma addr 0xf1c80000 at 0xffffa86a7000, size 462848
6. Source code location[edit | edit source]
6.1. User space[edit | edit source]
6.2. Kernel space[edit | edit source]
7. References[edit | edit source]
- ↑ Information about V4L2 Linux kernel framework on wikipedia.
- ↑ 2.0 2.1 2.2 Linux Media Infrastructure userspace API » Part I - Video for Linux API
- ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.5. Video Memory-To-Memory Interface
- ↑ 4.0 4.1 Media subsystem kernel internal API » 1. Video4Linux devices
- ↑ Linux Media Infrastructure userspace API » Part IV - Media Controller API