V4L2 video codec overview

Revision as of 13:01, 6 November 2023 by Registered User
Applicable for STM32MP25x lines

This article gives information about the Linux® V4L2 video codec framework.

1. Framework purpose[edit source]

The V4L2 Linux kernel framework[1] allows the control of video hardware codecs in order to decode and encode compressed video contents such as H264, VP8 or JPEG video bitstreams.

This could be typically used, with the help of other Linux multimedia frameworks, and applications, to play local videos, stream video contents, make a video recording or even remotely stream video compressed images from the camera sensor.

2. System overview[edit source]

VideoCodecOverview.png

2.1. Component description[edit source]

  • media-ctl (User space)

A V4L2 utility relying on Media Controller Linux kernel interface[2] aiming to configure, and link each sub devices composing the camera subsystem in order to configure it.

  • Application (User space)

Any application relying on V4L2 Linux kernel interface[3] or libv4l abstraction layer. GStreamer framework provides such application.

  • V4L2 utilities (User space)

A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor, and the camera interface. v4l2-ctl is one of the most useful utility.

  • V4L2 libraries (libv4l) (User space)

A set of libraries on top of the V4L2 Linux kernel interface[3] which abstract the kernel interface in order to simplify, keep compatibility or add some hooks between V4L-based applications, and the V4L2 kernel interface.

  • V4L2 core (Kernel space)

This layer represents the standard Linux kernel V4L2 Framework.

  • hantro_vpu (Kernel space)

This V4L2 Linux device driver handles the VDEC hardware block and the VENC hardware block.

  • VDEC (Hardware)

The Video DECoder hardware block.

  • VENC (Hardware)

The Video ENCoder hardware block.

2.2. APIs description[edit source]

The V4L2 userland API is documented in the Linux Media subsystem documentation[3]

More specifically, the V4L2 video memory-to-memory userland API is documented in the chapter[4]

The V4L2 kernel framework internal API is documented in the V4L2 Kernel Support section of the Linux kernel documentation[5]

The Media Controller API is documented in the Linux Media subsystem documentation[2]

3. Configuration[edit source]

3.1. Kernel configuration[edit source]

The STM32 hantro_vpu driver is enabled by default in STMicroelectronics deliveries.

Nevertheless, this could not be the case when using the upstream kernel version. In this case, the hantro_vpu V4L2 driver with its STM32MP25 backend can be enabled using Linux kernel menuconfig tool: 

[*] Device Drivers --->
    [*] Multimedia support --->
        [*] Media drivers ---> 
            [*] Media platform devices --->
                [*] Hantro VPU driver --->
                    [*] Hantro STM32MP25 support

3.2. Device tree configuration[edit source]

Refer to VDEC device tree configuration and VENC device tree configuration articles for Linux kernel device tree details.

4. How to use the framework[edit source]

The use cases described here are enabled using media-ctl, v4l2-ctl, gst-launch or gst-play command line utilities.

Warning white.png Warning
If Weston is configured with a "weston" user instead of a "root" user, please use the following command: 
 su -l "weston" -c "your_weston_command"

4.1. List the video devices and their capabilities[edit source]

List all the available video devices using --list-devices option: 

 v4l2-ctl --list-devices

st,stm32mp25-vdec-dec (platform: hantro-vpu):
        /dev/video0
        /dev/media0
st,stm32mp25-venc-enc (platform: hantro-vpu):
        /dev/video1
        /dev/media1

If several devices are available, use -d option after any v4l2-ctl commands to target a specific device. If -d option is not specified, /dev/video0 is targeted by default.

In order to have information on a specific device, use -D option.

/dev/video0 is the video decoder device: 

 v4l2-ctl -d /dev/video0 -D

Driver Info:
        Driver name      : hantro-vpu
        Card type        : st,stm32mp25-vdec-dec
        Bus info         : platform: hantro-vpu
        Driver version   : 6.1.28
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : hantro-vpu
        Model            : hantro-vpu
        Serial           : 
        Bus info         : platform: hantro-vpu
        Media version    : 6.1.28
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.1.28
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : st,stm32mp25-vdec-dec-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'st,stm32mp25-vdec-dec-proc' (Video Decoder): Data, Enabled, Immutable

/dev/video1 is the encoder device: 

 v4l2-ctl -d /dev/video1 -D

Driver Info:
        Driver name      : hantro-vpu
        Card type        : st,stm32mp25-venc-enc
        Bus info         : platform: hantro-vpu
        Driver version   : 6.1.28
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : hantro-vpu
        Model            : hantro-vpu
        Serial           : 
        Bus info         : platform: hantro-vpu
        Media version    : 6.1.28
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.1.28
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : st,stm32mp25-venc-enc-source
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Source
          Link 0x02000008: to remote pad 0x1000004 of entity 'st,stm32mp25-venc-enc-proc' (Video Encoder): Data, Enabled, Immutable



4.2. Get the supported compressed format, pixel format, resolution and framerate[edit source]

Use --list-formats-out-ext option to get the supported compressed formats and resolutions supported in input of decoder: 

 v4l2-ctl --list-formats-out-ext

ioctl: VIDIOC_ENUM_FMT
        Type: Video Output Multiplanar

        [0]: 'VP8F' (VP8 Frame, compressed)
                Size: Stepwise 96x96 - 4080x4080 with step 16/16
        [1]: 'S264' (H.264 Parsed Slice Data, compressed)
                Size: Stepwise 96x96 - 4080x4080 with step 16/16

Both VP8 and H264 bitstream decoding are supported up to 4080x4080.

Use --list-formats-ext option to get the supported pixel formats output by decoder: 

 v4l2-ctl -d /dev/video0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'NV12' (Y/UV 4:2:0)

Only NV12 (YUV420 semi-planar) is supported in output of decoder.

Same can be applied for encoder: 

 v4l2-ctl -d /dev/video1 --list-formats-out-ext

ioctl: VIDIOC_ENUM_FMT
        Type: Video Output Multiplanar

        [0]: 'NV12' (Y/UV 4:2:0)
        [1]: 'YM12' (Planar YUV 4:2:0 (N-C))
        [2]: 'NM12' (Y/UV 4:2:0 (N-C))
        [3]: 'YUYV' (YUYV 4:2:2)
        [4]: 'UYVY' (UYVY 4:2:2)
        [5]: 'RGBP' (16-bit RGB 5-6-5)
        [6]: 'XR24' (32-bit BGRX 8-8-8-8)
        [7]: 'BGR4' (32-bit BGRA/X 8-8-8-8)
        [8]: 'XB24' (32-bit RGBX 8-8-8-8)
        [9]: 'RX24' (32-bit XBGR 8-8-8-8)
        [10]: 'BX24' (32-bit XRGB 8-8-8-8)
        [11]: 'RGB4' (32-bit A/XRGB 8-8-8-8)

Encoder can be fed with YUV420 semi-planar and planar, YUV422 coplanar, RGB565 16 bits and RGB 32 bits pixels formats. 

 v4l2-ctl -d /dev/video1 --list-formats-ext

        Type: Video Capture Multiplanar

        [0]: 'JPEG' (JFIF JPEG, compressed)
                Size: Stepwise 96x32 - 8176x8176 with step 16/16
        [1]: 'VP8F' (VP8 Frame, compressed)
                Size: Stepwise 96x96 - 4080x4080 with step 16/16

Encoder can generate JPEG up to 8176 and VP8 bitstream up to 4080.


4.3. Downscale and crop[edit source]

Two features of DCMIPP can help to reduce resolution: downsize and crop, these two features are configured on postproc subdev.

Downscale is basic and consists in skipping 1 pixel over 2 in width and/or height, allowing to divide by 2 in width and/or height. The wanted resolution must be set on postproc subdev using compose property of sink pad respecting /2 ratio. Here is an example to get 320x240 frames from 640x480 input frames: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/320x240]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:RGB565_2X8_LE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/320x240]"

 gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=320,height=240, framerate=30/1 ! queue ! autovideosink
Info white.png Information
Please note that (top,left) argument of compose property is ignored by driver


Crop can be made at 16 pixels precision (output is rounded to 16 inferior multiple). The wanted area must be set using crop property of postproc subdev source pad. Here is an example to crop a 480x272 area centered inside the 640x480 input frame: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:RGB565_2X8_LE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"

 gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=480,height=272, framerate=30/1 ! queue ! autovideosink

Downscale and crop can be combined together. Here is an example to get small 320x176 frames from 1280x720 input frames: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/1280x720@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/640x360]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,92)/320x176]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:RGB565_2X8_LE/1280x720@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/640x360]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,92)/320x176]"

 gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=320,height=176, framerate=30/1 ! queue ! autovideosink

Refer to the selection chapter of the sub device interface documentation[6] to get more details on how to control scaling and crop.

4.4. Grab a raw frame[edit source]

Capture a VGA RGB565 raw frame on disk: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"

 v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Capture a QVGA RGB565 raw frame on disk: 


media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:RGB565_2X8_LE/320x240@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/320x240]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/320x240]"

 v4l2-ctl --set-fmt-video=width=320,height=240,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-320x240-rgb565.raw

In order to display it, this raw frame must be converted first to JPEG: 

 gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=153600 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg

Then weston-image utility can be used to display this JPEG file: 

 weston-image grab-640x480-rgb565.jpeg

4.5. Fullscreen preview[edit source]


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"

 gst-launch-1.0 v4l2src ! "video/x-raw, format=RGB16, width=480, height=272, framerate=(fraction)15/1" ! queue ! autovideosink -e
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:RGB565_2X8_LE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"

 gst-launch-1.0 v4l2src ! "video/x-raw, format=RGB16, width=480, height=272, framerate=(fraction)15/1" ! queue ! autovideosink -e

4.6. Take a picture in JPEG format[edit source]

The GC2145 sensor doesn't support JPEG frame format natively.

Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Capture a 5Mp JPEG: 


media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:JPEG_1X8/640x480@1/15 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:JPEG_1X8/2592x1944@1/15 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:JPEG_1X8/2592x1944]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:JPEG_1X8/2592x1944]"
Info white.png Information
Note the selection of 640x480@15 right before the wanted 2592x1944@1/15, this is because only 15 fps is supported by sensor for 5Mp resolution, so sensor must be switched first to 15 fps thanks to a resolution that support it, and only then 5Mp resolution can be selected. 
 v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=JPEG --stream-mmap --stream-count=1 --stream-skip=3 --stream-to=pic-5Mp.jpeg

Then display it: 

 weston-image pic-5Mp.jpeg

You can check the picture resolution using gst-typefind: 

 gst-typefind-1.0 pic-5Mp.jpeg

pic-5Mp.jpeg - image/jpeg, width=(int)2592, height=(int)1944, sof-marker=(int)0

4.7. Preview in YUV422[edit source]

Capture VGA YUV422 frames and display them using GStreamer : 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:YUYV8_2X8/640x480@1/15 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:YUYV8_2X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:YUYV8_2X8/640x480]"

 gst-launch-1.0 v4l2src ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)15/1" ! queue ! videoconvert ! queue ! autovideosink
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:YUYV8_2X8/640x480@1/15 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:YUYV8_2X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:YUYV8_2X8/640x480]"

 gst-launch-1.0 v4l2src ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)15/1" ! queue ! videoconvert ! queue ! autovideosink

4.8. Grab a raw-bayer frame[edit source]

Capture a 720p raw-bayer frame on disk. The GC2145 only currently support the RGGB format. 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:SRGGB8_1X8/1280x720@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB8_1X8/1280x720]"

 v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BA81 --stream-mmap --stream-count=1 --stream-to=grab-1280x720-bayer.raw

In order to display it, this raw-bayer frame must be converted first to JPEG: 

 gst-launch-1.0 filesrc location= grab-1280x720-bayer.raw blocksize=921600 ! "video/x-bayer, format=(string)rggb, width=(int)1280, height=(int)720, framerate=(fraction)30/1" ! bayer2rgb ! videoconvert ! jpegenc ! filesink location=grab-1280x720-bayer.jpeg

Then weston-image utility can be used to display this JPEG file: 

 weston-image grab-1280x720-bayer.jpeg
Click on the right to see/hide an example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Capture a 720p raw-bayer frame on disk: 


media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:SBGGR8_1X8/1280x720@1/15 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SBGGR8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SBGGR8_1X8/1280x720]"

 v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BA81 --stream-mmap --stream-count=1 --stream-to=grab-1280x720-bayer.raw

In order to display it, this raw-bayer frame must be converted first to JPEG: 

 gst-launch-1.0 filesrc location= grab-1280x720-bayer.raw blocksize=921600 ! "video/x-bayer, format=(string)bggr, width=(int)1280, height=(int)720, framerate=(fraction)15/1" ! bayer2rgb ! videoconvert ! jpegenc ! filesink location=grab-1280x720-bayer.jpeg

Then weston-image utility can be used to display this JPEG file: 

 weston-image grab-1280x720-bayer.jpeg

4.9. Raw-bayer capture with preview[edit source]

Camera subsystem is setup in order that 720p raw-bayer camera sensor output is reduced to 320x176 using downscale and crop: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:SRGGB8_1X8/1280x720@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/640x360]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,92)/320x176]"
Click on the right to see/hide an example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Camera subsystem is setup in order that 720p raw-bayer camera sensor output is reduced to 320x176 using downscale and crop: 


media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:SBGGR8_1X8/1280x720@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SBGGR8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SBGGR8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/640x360]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,92)/320x176]"

Then GStreamer is used to grab frames on disk while previewing: 

  gst-launch-1.0 v4l2src num-buffers=100 ! "video/x-bayer, format=rggb, width=320,height=176, framerate=(fraction)30/1" ! queue ! tee name=tee ! queue ! fpsdisplaysink name="capturerate" sync=false text-overlay=false video-sink="multifilesink location=320x180_%d-bayer.raw max-files=10"  tee. ! queue ! videorate max-rate=15 ! bayer2rgb ! queue ! videoconvert ! queue ! waylandsink sync=false -v

(for the example using OV5640 the format should be bggr within the gst-launch-1.0 command line)

Raw-bayer frames are captured on disk looping over 10 files to not saturate the disk: 

-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_90-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_91-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_92-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_93-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_94-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_95-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_96-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_97-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_98-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x180_99-bayer.raw

Note that capture occurs at 30 fps while preview is done at 15 fps to not overload system with software debayering, see trace below: 

/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 145, dropped: 0, current: 30.41, average: 30.63
/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 161, dropped: 0, current: 30.41, average: 30.60
/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 177, dropped: 0, current: 30.41, average: 30.59

5. How to trace and debug[edit source]

5.1. How to monitor[edit source]

5.1.1. Check of devicetree configuration[edit source]

Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about devicetree configuration: 


echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"'  -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh
echo "#!/bin/bash" > dtdump.sh;echo "find  \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh


rm devicetree.txt
echo "[devicetree]" >> devicetree.txt
echo "|-[dcmipp]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/dcmi | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[camera:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc/i2c*/camera*/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/i2c*/camera* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[mipi bridge:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc/i2c*/*mipi*/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/i2c*/*mipi* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "" >> devicetree.txt
cat devicetree.txt

[devicetree]
|-[dcmipp]
| |-dcmipp@5a000000/compatible=st,stm32mp13-dcmipp	73742c73746d33326d7031332d64636d69707000
| |-dcmipp@5a000000/clocks�	000000080000008f                        
| |-dcmipp@5a000000/resets5�	00000008000035c1                        
| |-dcmipp@5a000000/pinctrl-1=b	00000062                                
| |-dcmipp@5a000000/port/endpoint/hsync-active=	00000000                                
| |-dcmipp@5a000000/port/endpoint/vsync-active=	00000000                                
| |-dcmipp@5a000000/port/endpoint/remote-endpoint=c	00000063                                
| |-dcmipp@5a000000/port/endpoint/pclk-max-frequency='	07270e00                                
| |-dcmipp@5a000000/port/endpoint/bus-width=    00000008                                
| |-dcmipp@5a000000/port/endpoint/pclk-sample=	00000000                                
| |-dcmipp@5a000000/port/endpoint/phandle=>	0000003e                                
| |-dcmipp@5a000000/port/endpoint/name=endpoint	656e64706f696e7400                      
| |-dcmipp@5a000000/port/name=port	706f727400                              
| |-dcmipp@5a000000/clock-names=kclk	6b636c6b00                              
| |-dcmipp@5a000000/status=okay	6f6b617900                              
| |-dcmipp@5a000000/interrupts=O	000000000000004f00000004                
| |-dcmipp@5a000000/phandle=�	000000d4                                
| |-dcmipp@5a000000/reg=Z	5a00000000000400                        
| |-dcmipp@5a000000/pinctrl-0=a	00000061                                
| |-dcmipp@5a000000/name=dcmipp	64636d69707000                          
| |-dcmipp@5a000000/pinctrl-names=defaultsleep	64656661756c7400736c65657000            
|
|-[camera:ovti,ov5640]
| |-i2c@4c006000/camera@3c/compatible=ovti,ov5640	6f7674692c6f763536343000                
| |-i2c@4c006000/camera@3c/powerdown-gpios=/	0000002f0000000300000001                
| |-i2c@4c006000/camera@3c/DOVDD-supply=6	00000036                                
| |-i2c@4c006000/camera@3c/clocks=5	00000035                                
| |-i2c@4c006000/camera@3c/port/endpoint/data-lanes=	0000000100000002                        
| |-i2c@4c006000/camera@3c/port/endpoint/clock-lanes=	00000000                                
| |-i2c@4c006000/camera@3c/port/endpoint/phandle=�	000000a5                                
| |-i2c@4c006000/camera@3c/port/endpoint/name=endpoint	656e64706f696e7400                      
| |-i2c@4c006000/camera@3c/port/name=port	706f727400                              
| |-i2c@4c006000/camera@3c/clock-names=xclk	78636c6b00                              
| |-i2c@4c006000/camera@3c/status=disabled	64697361626c656400                      
| |-i2c@4c006000/camera@3c/reset-gpios=/	0000002f0000000400000001                
| |-i2c@4c006000/camera@3c/phandle=�	000000a4                                
| |-i2c@4c006000/camera@3c/reg=<	0000003c                                
| |-i2c@4c006000/camera@3c/name=camera	63616d65726100                          
|
|-[mipi bridge:st,st-mipid02]
| |-i2c@4c006000/stmipi@14/compatible=st,st-mipid02	73742c73742d6d69706964303200            
| |-i2c@4c006000/stmipi@14/clocks=;	0000003b                                
| |-i2c@4c006000/stmipi@14/VDDIN-supply=<	0000003c                                
| |-i2c@4c006000/stmipi@14/clock-names=xclk	78636c6b00                              
| |-i2c@4c006000/stmipi@14/ports/#address-cells=	00000001                                
| |-i2c@4c006000/stmipi@14/ports/#size-cells=	00000000                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/hsync-active=	00000000                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/vsync-active=	00000000                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/remote-endpoint=>	0000003e                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/bus-width=     00000008                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/pclk-sample=	00000000                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/phandle=c	00000063                                
| |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/name=endpoint	656e64706f696e7400                      
| |-i2c@4c006000/stmipi@14/ports/port@2/reg=	00000002                                
| |-i2c@4c006000/stmipi@14/ports/port@2/name=port	706f727400                              
| |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/remote-endpoint==	0000003d                                
| |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/data-lanes=	0000000100000002                        
| |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/phandle=7	00000037                                
| |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/lane-polarities=	000000000000000000000000                
| |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/name=endpoint	656e64706f696e7400                      
| |-i2c@4c006000/stmipi@14/ports/port@0/reg=	00000000                                
| |-i2c@4c006000/stmipi@14/ports/port@0/name=port	706f727400                              
| |-i2c@4c006000/stmipi@14/ports/name=ports	706f72747300                            
| |-i2c@4c006000/stmipi@14/status=okay	6f6b617900                              
| |-i2c@4c006000/stmipi@14/reset-gpios=/	0000002f0000000200000001                
| |-i2c@4c006000/stmipi@14/VDDE-supply=<	0000003c                                
| |-i2c@4c006000/stmipi@14/phandle=�	000000a6                                
| |-i2c@4c006000/stmipi@14/reg=	00000014                                
| |-i2c@4c006000/stmipi@14/name=stmipi	73746d69706900

5.1.2. Camera subsystem configuration debugging[edit source]

Refer to section #Get the topology of camera subsystem for basics on how to get and set camera subsystem configuration. Find below some debugging tips to help configuration handling.

5.1.2.1. subdevice configuration syntax error[edit source]

media-ctl utility returns an error when a configuration syntax mistake is made. Example here with erroneous entity name: 

 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]"
Unable to setup formats: Invalid argument (22)

In order to have details on the exact error, add -v option: 

 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" -v
no such entity "dcmipp_dump_postpr"
'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]
 ^

Another example with pad number mistake (only pad number 0 or 1 are allowed): 

 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" -v
No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1
 'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]
                        ^

Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8): 

 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" -v
Invalid pixel code 'BA81'
 'dcmipp_dump_postproc':0[fmt:BA81/640x480]
                         ^
5.1.2.2. subdevice negotiation[edit source]

Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of configuration it can handle. media-ctl -v option when setting configuration shows the effective subdevice configuration.

Here is an example with 5 fps framerate not accepted by the subdevice and 30 fps selected instead: 

 media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/5 field:none]" -v
Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0
Format set: RGB565_2X8_BE 640x480
Setting up frame interval 1/5 on pad gc2145 1-003c/0
Frame interval set: 1/30
Warning white.png Warning
No error or warning is returned by media-ctl, so take car to read back configuration with --set-v4l2 with -v or --get-v4l2 to check if it is fully accepted. Several negotiation rounds may be needed to find a fully compatible configuration.


Example with resolution negotiation (4000x4000 reduced to 2592x1944) 

 media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/4000x4000@1/15 field:none]" -v
Setting up format RGB565_2X8_BE 4000x4000 on pad gc2145 1-003c/0
Format set: RGB565_2X8_BE 1600x1200


Example with format negotiation (AYUV8_1X32 not supported, defaults to RGB565_2X8_LE): 

 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:AYUV8_1X32/640x480]" -v
Setting up format  AYUV8_1X32 640x480 on pad dcmipp_dump_postproc/0
Format set: RGB565_2X8_LE

5.2. How to trace[edit source]

5.2.1. V4L2 userland API tracing[edit source]

Tracing of V4L2 userland API[3] can be enabled using command: 

 echo 0x3 >  /sys/devices/platform/soc/5a000000.dcmipp/video4linux/video0/dev_debug

Traces are output in kernel log buffer: 

 dmesg
[  655.479016] video0: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_bytecap, bus=platform:dcmipp, version=0x00050f00, capabilities=0x852000011
[  655.481591] video0: VIDIOC_G_FMT: type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesperl0
[  655.481656] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, des'
[  655.481696] video0: VIDIOC_ENUM_FMT: index=1, type=vid-cap, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, des'
[  655.481732] video0: VIDIOC_ENUM_FMT: index=2, type=vid-cap, flags=0x0, pixelformat=YVYU little-endian (0x55595659), mbus_code=0x0000, des'
[  655.481767] video0: VIDIOC_ENUM_FMT: index=3, type=vid-cap, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, des'
[  655.481802] video0: VIDIOC_ENUM_FMT: index=4, type=vid-cap, flags=0x0, pixelformat=VYUY little-endian (0x59555956), mbus_code=0x0000, des'
[  655.481836] video0: VIDIOC_ENUM_FMT: index=5, type=vid-cap, flags=0x0, pixelformat=GREY little-endian (0x59455247), mbus_code=0x0000, des'
[  655.481873] video0: VIDIOC_ENUM_FMT: index=6, type=vid-cap, flags=0x0, pixelformat=BA81 little-endian (0x31384142), mbus_code=0x0000, des'
[  655.481909] video0: VIDIOC_ENUM_FMT: index=7, type=vid-cap, flags=0x0, pixelformat=GBRG little-endian (0x47524247), mbus_code=0x0000, des'
[  655.481944] video0: VIDIOC_ENUM_FMT: index=8, type=vid-cap, flags=0x0, pixelformat=GRBG little-endian (0x47425247), mbus_code=0x0000, des'
[  655.481981] video0: VIDIOC_ENUM_FMT: index=9, type=vid-cap, flags=0x0, pixelformat=RGGB little-endian (0x42474752), mbus_code=0x0000, des'
[  655.482017] video0: VIDIOC_ENUM_FMT: index=10, type=vid-cap, flags=0x1, pixelformat=JPEG little-endian (0x4745504a), mbus_code=0x0000, de'
[  655.482061] video0: VIDIOC_S_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.526124] video0: VIDIOC_REQBUFS: count=4, type=vid-cap, memory=mmap
[  655.526221] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526272] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526296] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526339] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526359] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=2, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526401] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526420] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=3, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526462] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.527192] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.536625] video0: VIDIOC_STREAMON: type=vid-cap
[  655.536701] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.892309] video0: VIDIOC_STREAMOFF: type=vid-cap
[  655.892435] video0: VIDIOC_STREAMOFF: type=vid-cap
[  655.901481] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap

5.2.2. V4L2 core framework tracing[edit source]

Tracing of the V4L2 core framework[5] can be enabled using commands: 


echo 0x3 > /sys/module/videobuf2_common/parameters/debug
echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug

Traces are output in kernel log buffer: 

 dmesg
[ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000
[ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000
[ 1218.656398] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 2, plane 0 offset 0x009a8000
[ 1218.669843] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 3, plane 0 offset 0x00e7c000
[ 1218.677503] videobuf2_common: [cap-7b50f8bf] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each
[ 1218.688361] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 0, plane 0 successfully mapped
[ 1218.696744] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 1, plane 0 successfully mapped
[ 1218.705291] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 2, plane 0 successfully mapped
[ 1218.712487] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 3, plane 0 successfully mapped
[ 1218.722474] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.730528] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.738153] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.746012] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.756237] videobuf2_common: [cap-7b50f8bf] vb2_core_streamon: successful
[ 1218.761969] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.822167] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.828383] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued
[ 1218.838812] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.846549] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.855624] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.862118] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued
[ 1218.872406] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.880029] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.889460] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.896756] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 2, state: dequeued
[ 1218.905383] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.912309] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.923080] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.930392] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 3, state: dequeued
[ 1218.944499] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.952210] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.959921] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.968800] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 0
[ 1218.978419] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 1
[ 1218.988204] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 2
[ 1218.997501] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 3

5.2.3. DCMIPP V4L2 kernel driver tracing[edit source]

DCMIPP dynamic debug traces[7] can be enabled using command: 


echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control
echo "module dcmipp_bytecap +p" > /sys/kernel/debug/dynamic_debug/control
echo "module dcmipp_byteproc +p" > /sys/kernel/debug/dynamic_debug/control
echo "module dcmipp_parallel +p" > /sys/kernel/debug/dynamic_debug/control

Here is an example with a 5Mp jpeg capture:

media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]" 

  v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw

 dmesg
[  236.968937] dcmipp-parallel dcmipp-parallel.1.auto: dcmipp_parallel: format update: old:640x480 (0x1007, 8, 1, 2, 1) new:640x480 (0x1007, 3, 0, 0, 0)
[  236.987292] dcmipp-parallel dcmipp-parallel.1.auto: dcmipp_parallel: format update: old:640x480 (0x1007, 3, 0, 0, 0) new:640x480 (0x1008, 3, 0, 0, 0)
[  246.673178] dcmipp-bytecap dcmipp-bytecap.3.auto: dcmipp_dump_capture: format update: old:640x480 (0x50424752, 8, 1, 2, 0) new:640x480 (0x50424752, 8, 1, 2, 0)
[  246.673704] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup queue, count=4, size=614400
[  246.691053] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [0] phy=0xd4a00000 size=614400
[  246.691114] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [1] phy=0xd4b00000 size=614400
[  246.691136] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [2] phy=0xd4c00000 size=614400
[  246.691156] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [3] phy=0xd4d00000 size=614400
[  246.691189] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [0] cdcc4929 phy=0xd4a00000
[  246.691208] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [1] 8cfa694d phy=0xd4b00000
[  246.691221] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [2] 57f4cf64 phy=0xd4c00000
[  246.691233] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [3] 9a8d0e1c phy=0xd4d00000
[  246.691397] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0FCTCR 0x00000003
[  246.691415] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0FCTCR 0x00000000
[  246.691425] dcmipp-byteproc dcmipp-byteproc.2.auto: SET DCMIPP_P0FCTCR 0x00000000
[  246.691435] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0FCTCR 0x00000000
[  246.691445] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0PPCR 0x00000180
[  246.691455] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0PPCR 0x00000000
[  246.691465] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0PPCR 0x00000400
[  246.691474] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0PPCR 0x00000000
[  246.691484] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0SCSTR 0x00000000
[  246.691493] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0SCSZR 0x00000000
[  246.691504] dcmipp-byteproc dcmipp-byteproc.2.auto: crop to 640x480
[  246.691514] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0SCSTR 0x00000000
[  246.691524] dcmipp-byteproc dcmipp-byteproc.2.auto: WR  DCMIPP_P0SCSZR 0x81e00140
[  246.691535] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_dump_postproc" is started
[  246.691547] dcmipp-parallel dcmipp-parallel.1.auto: WR  DCMIPP_PRCR 0x02220000
[  246.691557] dcmipp-parallel dcmipp-parallel.1.auto: SET DCMIPP_PRCR 0x00004000
[  246.691567] dcmipp-parallel dcmipp-parallel.1.auto: WR  DCMIPP_PRCR 0x02224000
[  246.691576] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_parallel" is started
[  246.693135] dcmipp-bytecap dcmipp-bytecap.3.auto: "st-mipid02 1-0014" is started
[  246.832561] dcmipp-bytecap dcmipp-bytecap.3.auto: "gc2145 1-003c" is started
[  246.832598] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_P0FSCR 0x80000000
[  246.832619] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0FSCR 0x80000000
[  246.832635] dcmipp-bytecap dcmipp-bytecap.3.auto: Start with next [0] cdcc4929 phy=0xd4a00000
[  246.832661] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0PPM0AR1 0xd4a00000
[  246.832676] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [0] cdcc4929 phy=0xd4a00000
[  246.832695] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0DCLMTR 0x80025800
[  246.832709] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_P0FCTCR 0x00000008
[  246.832724] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0FCTCR 0x00000008
[  246.832738] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_CMIER 0x00008600
[  246.832753] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMIER 0x00008600
[  247.094680] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR2 0x00000400
[  247.094723] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMFCR 0x00000400
[  247.094759] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0PPM0AR1 0xd4b00000
[  247.094777] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [1] 8cfa694d phy=0xd4b00000
[  247.128387] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR2 0x00000300
[  247.128430] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMFCR 0x00000300
[  247.128469] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_P0DCCNTR 0x00096000
[  247.128498] dcmipp-bytecap dcmipp-bytecap.3.auto: Done  [0] cdcc4929 phy=0xd4a00000
[  247.129127] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR2 0x00000500
[  247.129163] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMFCR 0x00000500
[  247.129357] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0PPM0AR1 0xd4c00000
[  247.129383] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [2] 57f4cf64 phy=0xd4c00000
[  247.149857] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [0] cdcc4929 phy=0xd4a00000
[  247.150021] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_dump_postproc" is stopped
[  247.150040] dcmipp-parallel dcmipp-parallel.1.auto: CLR DCMIPP_PRCR 0x00004000
[  247.150053] dcmipp-parallel dcmipp-parallel.1.auto: WR  DCMIPP_PRCR 0x02220000
[  247.150074] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR2 0x00000300
[  247.150085] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMFCR 0x00000300
[  247.150130] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_P0DCCNTR 0x0005f000
[  247.150147] dcmipp-bytecap dcmipp-bytecap.3.auto: Done  [1] 8cfa694d phy=0xd4b00000
[  247.156441] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_parallel" is stopped
[  247.156965] dcmipp-bytecap dcmipp-bytecap.3.auto: "st-mipid02 1-0014" is stopped
[  247.157010] dcmipp-bytecap dcmipp-bytecap.3.auto: "gc2145 1-003c" is stopped
[  247.157028] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_CMIER 0x00008600
[  247.157043] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_CMIER 0x00000000
[  247.157053] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_P0FCTCR 0x00000008
[  247.157062] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0FCTCR 0x00000000
[  247.157073] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_P0FSCR 0x80000000
[  247.157083] dcmipp-bytecap dcmipp-bytecap.3.auto: WR  DCMIPP_P0FSCR 0x00000000
[  247.157100] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_PRSR 0x00030000
[  247.157111] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_PRSR]  =0x00030000
[  247.157120] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_P0SR 0x00000007
[  247.157129] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_P0SR] =0x00000007
[  247.157139] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_P0DCCNTR 0x0005f000
[  247.157148] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_P0DCCNTR]=0x0005f000
[  247.157157] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR1 0x00000003
[  247.157166] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_CMSR1] =0x00000003
[  247.157175] dcmipp-bytecap dcmipp-bytecap.3.auto: RD  DCMIPP_CMSR2 0x00000000
[  247.157184] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_CMSR2] =0x00000000
[  247.157198] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=0 (overrun=0, limit=0, nactive=0), underrun=0, vsync=2, frame=2, buffers=2, it=4

5.3. How to debug[edit source]

5.3.1. Errors[edit source]

Errors are unconditionally traced in kernel log: 

 dmesg
[ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4


5.3.2. Memory tracking[edit source]

Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through DMA backend. Those allocations can be traced using: 


echo "module dma_contiguous +p" > /sys/kernel/debug/dynamic_debug/control
echo "module videobuf2_dma_contig +p" > /sys/kernel/debug/dynamic_debug/control

Here is the trace after a VGA YUV422 capture. 

[11311.617688] vb2_dc_mmap: mapped dma addr 0xf1900000 at 0xb3b6a000, size 614400
[11311.617986] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xb3ad4000, size 614400
[11311.618071] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xb3a3e000, size 614400
[11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400

4 frames of VGA YUV422 frames: 640x480x2=614400 bytes

6. Source code location[edit source]

6.1. User space[edit source]

6.2. Kernel space[edit source]

7. References[edit source]

  1. ↑ Information about V4L2 Linux kernel framework on wikipedia.
  2. ↑ 2.0 2.1 Linux Media Infrastructure userspace API » Part IV - Media Controller API
  3. ↑ 3.0 3.1 3.2 3.3 Linux Media Infrastructure userspace API » Part I - Video for Linux API
  4. ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.5. Video Memory-To-Memory Interface
  5. ↑ 5.0 5.1 Media subsystem kernel internal API » 1. Video4Linux devices
  6. ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition
  7. ↑ How to use the kernel dynamic debug



Linux® is a registered trademark of Linus Torvalds.

Video 4 Linux® version 2

Advanced Video Coding (AVC) or H.264 or MPEG-4 Part 10, is a video compression standard (see https://en.wikipedia.org/wiki/H264) - NEW

open and royalty-free video compression format (see https://en.wikipedia.org/wiki/VP8) - NEW

Video decoder - NEW

Video encoder - NEW

Application programming interface

fourcc of YUV420 semi-planar pixel format

Digital Camera memory interface pixel processor

Direct memory access

Retrieved from "https://wiki.st.com/stm32mpu/index.php?title=V4L2_video_codec_overview&oldid=96337"