This article gives information about the Linux® V4L2 camera framework.
1. Framework purpose[edit source]
The V4L2 Linux kernel framework[1] allows the control of both an external camera sensor, and the camera interface. This allows the capture of raw frames in various pixel formats or encoded stream data such as JPEG.
This could be typically used, with the help of other Linux multimedia frameworks, and applications, to take snapshot, to make preview, to make a video recording or even remotely stream images from the camera sensor.
2. System overview[edit source]
2.1. Component description[edit source]
- media-ctl (User space)
A V4L2 utility relying on Media Controller Linux kernel interface[2] aiming to configure, and link each sub devices composing the camera subsystem in order to configure it.
- Application (User space)
Any application relying on V4L2 Linux kernel interface[3] or libv4l abstraction layer. GStreamer framework provides such application.
- V4L2 utilities (User space)
A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor, and the camera interface. v4l2-ctl is one of the most useful utility.
- V4L2 libraries (libv4l) (User space)
A set of libraries on top of the V4L2 Linux kernel interface[3] which abstract the kernel interface in order to simplify, keep compatibility or add some hooks between V4L-based applications, and the V4L2 kernel interface.
- V4L2 core (Kernel space)
This layer represents the standard Linux kernel V4L2 Framework.
- stm32-dcmipp (Kernel space)
This V4L2 DCMIPP Linux device driver handles the DCMIPP hardware block.
- Camera sensor X driver (Kernel space)
This V4L2 Linux device driver handles the camera sensor X external peripheral, handles some GPIOs lines, and potentially power supplies to power-up/down the camera sensor. The communication with camera sensor is done through the i2c bus.
- DCMIPP (Hardware)
The Digital Camera Memory Interface Pixel Processor hardware block.
- Camera sensor X (Hardware)
The camera sensor external peripheral.
2.2. APIs description[edit source]
The V4L2 userland API is documented in the Linux Media subsystem documentation[3]
The V4L2 kernel framework internal API is documented in the V4L2 Kernel Support section of the Linux kernel documentation[4]
The Media Controller API is documented in the Linux Media subsystem documentation[2]
3. Configuration[edit source]
3.1. Kernel configuration[edit source]
The STM32 DCMIPP camera interface and the camera sensor are enabled by default in STMicroelectronics deliveries.
Nevertheless, this is not the case when using the upstream kernel version. In this case, the DCMIPP V4L2 driver can be enabled using Linux kernel menuconfig tool:
Device Drivers --->
<M> Multimedia support --->
Media drivers --->
[*] Media platform devices --->
<M> STM32 Digital Camera Memory Interface Pixel Processor (DCMIPP) support
![]() |
Double-check the links below when the STM32MP257x-EV1_-_hardware_description board article is ready |
The external camera sensor connected to the camera interface must also be enabled, here is an example with the IMX335 SONY camera sensor located on the MB1854 camera daughter board[5] connected to the CN4 CSI camera connector[6] of the STM32MP257F-EV1 Evaluation board :
Device Drivers ---> <M> Multimedia support ---> Media ancillary drivers ---> [*] Camera sensor devices ---> <M> Sony IMX335 sensor support
3.2. Device tree configuration[edit source]
Refer to DCMIPP device tree configuration article for a complete view of DCMIPP and sensor configuration thanks to Linux kernel device tree mechanism.
4. How to use the framework[edit source]
The use cases described here are enabled using media-ctl, v4l2-ctl, gst-launch or gst-play command line utilities.
4.1. List the video devices and their capabilities[edit source]
List all the available video devices using --list-devices option:
v4l2-ctl --list-devices
dcmipp_bytecap (platform:dcmipp): /dev/video2 /dev/video3 /dev/video4 /dev/video5 /dev/media2
If several devices are available, use -d option after any v4l2-ctl commands to target a specific device. If -d option is not specified, /dev/video0 is targeted by default.
In order to have information on a specific device, use -D option:
v4l2-ctl -d /dev/video3 -D
root@stm32mp25:~# v4l2-ctl -d /dev/video3 -D Driver Info: Driver name : dcmipp Card type : dcmipp_pixelcap Bus info : platform:dcmipp Driver version : 6.1.28 Capabilities : 0x85200001 Video Capture Read/Write Streaming Extended Pix Format Device Capabilities Device Caps : 0x05200001 Video Capture Read/Write Streaming Extended Pix Format Media Driver Info: Driver name : dcmipp Model : DCMIPP MDEV Serial : Bus info : platform:dcmipp Media version : 6.1.28 Hardware revision: 0x00000000 (0) Driver version : 6.1.28 Interface Info: ID : 0x03000015 Type : V4L Video Entity Info: ID : 0x00000013 (19) Name : dcmipp_main_capture Function : V4L2 I/O Pad 0x01000014 : 0: Sink Link 0x0200002c: from remote pad 0x1000012 of entity 'dcmipp_main_postproc' (Video Pixel Formatter): Data, Enabled, Immutable
4.2. Get the topology of camera subsystem[edit source]
Print the topology of camera subsystem using -p media-ctl option:
media-ctl -d /dev/media2 -p
Media controller API version 6.1.28 Media device information ------------------------ driver dcmipp model DCMIPP MDEV serial bus info platform:dcmipp hw revision 0x0 driver version 6.1.28 Device topology - entity 1: dcmipp_parallel (2 pads, 3 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709] pad1: Source [fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709] -> "dcmipp_dump_postproc":0 [] -> "dcmipp_main_isp":0 [] -> "dcmipp_aux_postproc":0 [] - entity 4: dcmipp_dump_postproc (2 pads, 3 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:SRGGB10_1X10/2592x1940@1/30 field:none colorspace:rec709 compose.bounds:(0,0)/2592x1940 compose:(0,0)/2592x1940] <- "dcmipp_parallel":1 [] <- "stm32_csi2host.48020000.csi2hos":1 [ENABLED] pad1: Source [fmt:SRGGB10_1X10/2592x1940@1/30 field:none colorspace:rec709 crop.bounds:(0,0)/2592x1940 crop:(0,0)/2592x1940] -> "dcmipp_dump_capture":0 [ENABLED,IMMUTABLE] - entity 7: dcmipp_dump_capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2 pad0: Sink <- "dcmipp_dump_postproc":1 [ENABLED,IMMUTABLE] - entity 11: dcmipp_main_isp (4 pads, 5 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: Sink [fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709 crop.bounds:(0,0)/2592x1940 crop:(0,0)/2592x1940 compose:(0,0)/2592x1940] <- "dcmipp_parallel":1 [] <- "stm32_csi2host.48020000.csi2hos":1 [ENABLED] pad1: Source [fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709] -> "dcmipp_main_postproc":0 [ENABLED,IMMUTABLE] pad2: Source [fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709] -> "dcmipp_aux_postproc":0 [] pad3: Source [fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709] -> "dcmipp_main_isp_stat_capture":0 [ENABLED,IMMUTABLE] - entity 16: dcmipp_main_postproc (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev3 pad0: Sink [fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709 crop.bounds:(0,0)/2592x1940 crop:(0,0)/2592x1940 compose:(0,0)/640x480] <- "dcmipp_main_isp":1 [ENABLED,IMMUTABLE] pad1: Source [fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709] -> "dcmipp_main_capture":0 [ENABLED,IMMUTABLE] - entity 19: dcmipp_main_capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video3 pad0: Sink <- "dcmipp_main_postproc":1 [ENABLED,IMMUTABLE] - entity 23: dcmipp_aux_postproc (2 pads, 4 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev4 pad0: Sink [fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709 crop.bounds:(0,0)/2592x1940 crop:(0,0)/2592x1940 compose:(0,0)/2592x1940] <- "dcmipp_parallel":1 [] <- "dcmipp_main_isp":2 [] <- "stm32_csi2host.48020000.csi2hos":1 [ENABLED] pad1: Source [fmt:RGB565_2X8_LE/2592x1940 field:none] -> "dcmipp_aux_capture":0 [ENABLED,IMMUTABLE] - entity 26: dcmipp_aux_capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video4 pad0: Sink <- "dcmipp_aux_postproc":1 [ENABLED,IMMUTABLE] - entity 30: dcmipp_main_isp_stat_capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video5 pad0: Sink <- "dcmipp_main_isp":3 [ENABLED,IMMUTABLE] - entity 52: stm32_csi2host.48020000.csi2hos (9 pads, 4 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev5 pad0: Sink [fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none] <- "imx335 0-001a":0 [ENABLED,IMMUTABLE] pad1: Source [fmt:SRGGB10_1X10/2592x1940] -> "dcmipp_dump_postproc":0 [ENABLED] -> "dcmipp_main_isp":0 [ENABLED] -> "dcmipp_aux_postproc":0 [ENABLED] pad2: Source [fmt:UYVY8_2X8/0x0] pad3: Source [fmt:UYVY8_2X8/0x0] pad4: Source [fmt:UYVY8_2X8/0x0] pad5: Source [fmt:UYVY8_2X8/0x0] pad6: Source [fmt:UYVY8_2X8/0x0] pad7: Source [fmt:UYVY8_2X8/0x0] pad8: Source [fmt:UYVY8_2X8/0x0] - entity 68: imx335 0-001a (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev6 pad0: Source [fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none] -> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
This gives the current links between camera subdevices, and each subdevice current pads configuration such as, format, resolution, framerate.
This topology can also be displayed in a human readable graphical form thanks to --print-dot option:
media-ctl -d /dev/media2 --print-dot > graph.dot
Retrieve then this .dot file into your host PC in order to convert it to .png:
dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png
This picture shows the 2 possible camera interface: MIPI CSI-2 or parallel, and the 3 pipes topology: dump pipe, main pipe and auxiliary pipe:
- dump pipe allows to directly read without any processing the camera sensor data whatever those data are: raw-bayer, RGB, YUV, JPEG or even image statistic data. The dump pipe output camera sensor specific data can be read from /dev/video2 capture node.
- main pipe is the most featured DCMIPP pipe, including frame skipping, debayering, image correction thanks to DCMIPP ISP, scaling, color conversion, ... The main pipe output pixel data can be read from /dev/video3 capture node while the image statistic data can be read from /dev/video5 capture node. Refer to How to use the DCMIPP ISP article to get more details on how to hande image statistic data.
- auxilliary pipe can perform limited operations such as scaling and color conversion with some restrictions regarding to main pipe. It can also be dynamically linked to the output of ISP in order to inherit of image correction. The auxiliary pipe output pixel data can be read from /dev/video4 capture node.
4.3. Camera subsystem setup[edit source]
In order to be able to capture frames from video device node with v4l2-ctl, GStreamer or any other V4L2 application, the camera subsystem must be first configured. To do so use media-ctl --set-v4l2 command giving the subdevice name & pad, the desired format, resolution and framerate:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]"
The configuration must be done from source to sink. Below is a configuration allowing to capture RGB565 640x480 frames:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
Camera sensor is configured in 5Mp raw-bayer 10bits at 30 fps -which is the only one configuration supported by the IMX335 for now-, then DCMIPP is configured to convert to RGB565 and downscale to 640x480. The configuration can be read back displaying the topology of graph, see #Get the topology of camera subsystem. Then, any application that read frames from V4L2 video device node can be executed:
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true
![]() |
At this stage you can observe that preview image is very dark and greenish, see next chapter to perform image correction. |
4.4. Image correction[edit source]
In order to correct image coming from raw-bayer camera sensor using DCMIPP ISP, use STMicroelectronics isp command line utility with -w option while capture is running:
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true &
sleep 1
/usr/local/demo/application/camera/bin/isp -w
sleep 1
kill %1
Image correction parameters are kept from a capture to another one, so you don't have to recall isp utility if ambient light conditions have not changed. Anyway you can call isp -w in a loop in background to regularly correct image. More details on image correction can be found in the dedicated article How to use the DCMIPP ISP.
4.5. Controlling camera sensor[edit source]
In order to control camera sensor, its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, ie the entity having no sink pad:
- entity 68: imx335 0-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev6
pad0: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none]
-> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
Here it is /dev/v4l-subdev6.
Use then v4l2-ctl with -L option to get the list of supported controls:
v4l2-ctl -d /dev/v4l-subdev6 -L
User Controls exposure 0x00980911 (int) : min=1 max=4491 step=1 default=1608 value=1608 Image Source Controls vertical_blanking 0x009e0901 (int) : min=2560 max=133060 step=1 default=2560 value=2560 horizontal_blanking 0x009e0902 (int) : min=0 max=1048575 step=1 default=342 value=342 flags=read-only analogue_gain 0x009e0903 (int) : min=0 max=240 step=1 default=0 value=0 Image Processing Controls link_frequency 0x009f0901 (intmenu): min=0 max=0 default=0 value=0 (594000000 0x2367b880) flags=read-only 0: 594000000 (0x2367b880) pixel_rate 0x009f0902 (int64) : min=396000000 max=396000000 step=1 default=396000000 value=396000000 flags=read-only
![]() |
"value=" field returns the current value of the control |
The control value can be changed thanks to --set-ctrl option, for example:
v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=10
The control value can be changed dynamically. In the following example, the analog gain is increased while preview is running:
- Start preview in background
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true -e > /dev/null 2>&1 &
- Then increase gain every second
v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=0;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=20;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=40;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=0;killall gst-launch-1.0
4.6. Set the camera sensor pixel format, resolution and framerate[edit source]
In order to control camera sensor, its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, ie the entity having no sink pad:
- entity 68: imx335 0-001a (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev6 pad0: Source [fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none] -> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
Here it is /dev/v4l-subdev6.
Camera sensor only supports a discrete set of formats, resolutions and framerates that can be get thanks to v4l2-ctl options --list-subdev-mbus-codes, --list-subdev-framesizes and --list-subdev-frameintervals:
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-mbus-codes
or
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-mbus-codes <pad number> ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0) 0x300f: MEDIA_BUS_FMT_SRGGB10_1X10
Frame size might depend on frame format, so precise it when asking for frame size:
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-framesizes pad=0,code=0x300f ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0) Size Range: 2592x1940 - 2592x1940
Framerate depends on resolution and format, so precise them when asking for frame intervals:
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-frameintervals pad=0,width=2592,height=1940,code=0x300f ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0) Interval: 0.033s (30.000 fps)
Now that camera sensor capabilities are known, the wanted configuration can be set following instructions described in chapter camera subsystem setup:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]"
4.7. Downscale and crop[edit source]
Three features of DCMIPP can help to reduce resolution: decimation, downsize and crop.
Main pipe is able to crop and downscale. To do so postproc subdev must be configured accordingly. The wanted resolution must be set on postproc subdev using compose property of sink pad. Here is an example to get 640x480 frames from 2592x1940 input frames coming from camera sensor:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640, height=480, framerate=30/1 ! queue ! autovideosink
![]() |
Please note that (top,left) argument of compose property is ignored by driver |
The wanted crop area must be set using crop property of postproc subdev sink pad.
Here is an example to crop a 640x480 area centered inside the 2592x1940 input frame coming from camera sensor:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink
Downscale and crop can be combined together. Here is an example to get 640x480 frames from 2592x1940 input frames cropped to 1292x970:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/1292x970]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink
Refer to the selection chapter of the sub device interface documentation[7] to get more details on how to control scaling and crop.
4.8. Grab a raw frame[edit source]
Capture a VGA RGB565 raw frame on disk:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
v4l2-ctl -d /dev/video3 --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw
In order to display it, this raw frame must be converted first to JPEG:
gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=614400 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg
Then weston-image utility can be used to display this JPEG file:
weston-image grab-640x480-rgb565.jpeg
4.9. Fullscreen preview[edit source]
Camera preview demo application -started by clicking on camera icon menu- is a good example of script based command lines which combines camera subsystem configuration using media-ctl, preview using GStreamer and automatic image correction using isp utility in background. You can manually start this demo script using command line:
/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh
With GPU
/home/root
mediadev=/dev/media2
sensorsubdev="imx335 0-001a"
sensordev= /dev/v4l-subdev6
interfacesubdev=stm32_csi2host.48020000.csi2hos
sensorbuscode=SRGGB10_1X10
SENSORWIDTH=2592
SENSORHEIGHT=1940
Mediacontroller graph:
media-ctl -d /dev/media2 --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]"
media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]"
media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
SCREEN_WIDTH= 1024
SCREEN_HEIGHT=600
GStreamer graph:
v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640, height=480 ! queue ! gtkwaylandsink name=gtkwsink
media-ctl camera subsystem configuration and GStreamer preview commands are displayed to ease further debugging and prototyping using copy/paste:
media-ctl -d /dev/media2 --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=RGB16, width=640, height=480 ! queue ! gtkwaylandsink name=gtkwsink
4.10. Take a picture in JPEG format[edit source]
The IMX335 camera sensor doesn't support JPEG frame format natively, but you can use hardware JPEG video encoder to encode raw camera sensor to JPEG.
Capture a 5Mp YUV422 raw frame:
media-ctl -d /dev/media2 --set-v4l2 "imx335 0-001a:0[fmt:SRGGB10_1X10/2592x1940@1/30]" media-ctl -d /dev/media2 --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/2592x1940]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/2592x1940]"
v4l2-ctl -d /dev/video3 --set-fmt-video=width=2592,height=1944,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-skip=3 --stream-to=pic-5Mp.yuyv
Convert to JPEG
gst-launch-1.0 filesrc location= pic-5Mp.yuyv blocksize=10077696 ! "video/x-raw, format=YUY2, width=2592, height=1940, framerate=30/1, colorimetry=2:0:0:0" ! v4l2jpegenc ! filesink location=pic-5Mp.jpeg
Then display it:
weston-image pic-5Mp.jpeg
You can check the picture resolution using gst-typefind:
gst-typefind-1.0 pic-5Mp.jpeg pic-5Mp.jpeg - image/jpeg, width=(int)2592, height=(int)1944, sof-marker=(int)0
4.11. Preview in YUV422[edit source]
Capture VGA YUV422 frames and display them using GStreamer :
media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:YUYV8_2X8/640x480@1/15 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:YUYV8_2X8/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:YUYV8_2X8/640x480]"
gst-launch-1.0 v4l2src ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)15/1" ! queue ! videoconvert ! queue ! autovideosink
media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:YUYV8_2X8/640x480@1/15 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:YUYV8_2X8/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:YUYV8_2X8/640x480]" gst-launch-1.0 v4l2src ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)15/1" ! queue ! videoconvert ! queue ! autovideosink |
4.12. Grab a raw-bayer frame[edit source]
Capture a 720p raw-bayer frame on disk. The GC2145 only currently support the RGGB format.
media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:SRGGB8_1X8/1280x720@1/30 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SRGGB8_1X8/1280x720]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':1[fmt:SRGGB8_1X8/1280x720]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SRGGB8_1X8/1280x720]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB8_1X8/1280x720]"
v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BA81 --stream-mmap --stream-count=1 --stream-to=grab-1280x720-bayer.raw
In order to display it, this raw-bayer frame must be converted first to JPEG:
gst-launch-1.0 filesrc location= grab-1280x720-bayer.raw blocksize=921600 ! "video/x-bayer, format=(string)rggb, width=(int)1280, height=(int)720, framerate=(fraction)30/1" ! bayer2rgb ! videoconvert ! jpegenc ! filesink location=grab-1280x720-bayer.jpeg
Then weston-image utility can be used to display this JPEG file:
weston-image grab-1280x720-bayer.jpeg
Capture a 720p raw-bayer frame on disk: media-ctl -d /dev/media0 --set-v4l2 "'ov5640 1-003c':0[fmt:SBGGR8_1X8/1280x720@1/15 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:SBGGR8_1X8/1280x720]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SBGGR8_1X8/1280x720]" v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BA81 --stream-mmap --stream-count=1 --stream-to=grab-1280x720-bayer.raw In order to display it, this raw-bayer frame must be converted first to JPEG: gst-launch-1.0 filesrc location= grab-1280x720-bayer.raw blocksize=921600 ! "video/x-bayer, format=(string)bggr, width=(int)1280, height=(int)720, framerate=(fraction)15/1" ! bayer2rgb ! videoconvert ! jpegenc ! filesink location=grab-1280x720-bayer.jpeg Then weston-image utility can be used to display this JPEG file: weston-image grab-1280x720-bayer.jpeg |
5. How to trace and debug[edit source]
5.1. How to monitor[edit source]
5.1.1. Check of devicetree configuration[edit source]
Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about devicetree configuration:
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh echo "#!/bin/bash" > dtdump.sh;echo "find \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh
rm devicetree.txt echo "[devicetree]" >> devicetree.txt echo "|-[dcmipp]" >> devicetree.txt ./dtdump.sh /proc/device-tree/soc/dcmi | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt echo "|" >> devicetree.txt echo "|-[camera:" | tr -d "\n" >> devicetree.txt cat /proc/device-tree/soc/i2c*/camera*/compatible >> devicetree.txt echo "]" >> devicetree.txt ./dtdump.sh /proc/device-tree/soc/i2c*/camera* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt echo "|" >> devicetree.txt echo "|-[mipi bridge:" | tr -d "\n" >> devicetree.txt cat /proc/device-tree/soc/i2c*/*mipi*/compatible >> devicetree.txt echo "]" >> devicetree.txt ./dtdump.sh /proc/device-tree/soc/i2c*/*mipi* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt echo "" >> devicetree.txt cat devicetree.txt
[devicetree] |-[dcmipp] | |-dcmipp@5a000000/compatible=st,stm32mp13-dcmipp 73742c73746d33326d7031332d64636d69707000 | |-dcmipp@5a000000/clocks� 000000080000008f | |-dcmipp@5a000000/resets5� 00000008000035c1 | |-dcmipp@5a000000/pinctrl-1=b 00000062 | |-dcmipp@5a000000/port/endpoint/hsync-active= 00000000 | |-dcmipp@5a000000/port/endpoint/vsync-active= 00000000 | |-dcmipp@5a000000/port/endpoint/remote-endpoint=c 00000063 | |-dcmipp@5a000000/port/endpoint/pclk-max-frequency=' 07270e00 | |-dcmipp@5a000000/port/endpoint/bus-width= 00000008 | |-dcmipp@5a000000/port/endpoint/pclk-sample= 00000000 | |-dcmipp@5a000000/port/endpoint/phandle=> 0000003e | |-dcmipp@5a000000/port/endpoint/name=endpoint 656e64706f696e7400 | |-dcmipp@5a000000/port/name=port 706f727400 | |-dcmipp@5a000000/clock-names=kclk 6b636c6b00 | |-dcmipp@5a000000/status=okay 6f6b617900 | |-dcmipp@5a000000/interrupts=O 000000000000004f00000004 | |-dcmipp@5a000000/phandle=� 000000d4 | |-dcmipp@5a000000/reg=Z 5a00000000000400 | |-dcmipp@5a000000/pinctrl-0=a 00000061 | |-dcmipp@5a000000/name=dcmipp 64636d69707000 | |-dcmipp@5a000000/pinctrl-names=defaultsleep 64656661756c7400736c65657000 | |-[camera:ovti,ov5640] | |-i2c@4c006000/camera@3c/compatible=ovti,ov5640 6f7674692c6f763536343000 | |-i2c@4c006000/camera@3c/powerdown-gpios=/ 0000002f0000000300000001 | |-i2c@4c006000/camera@3c/DOVDD-supply=6 00000036 | |-i2c@4c006000/camera@3c/clocks=5 00000035 | |-i2c@4c006000/camera@3c/port/endpoint/data-lanes= 0000000100000002 | |-i2c@4c006000/camera@3c/port/endpoint/clock-lanes= 00000000 | |-i2c@4c006000/camera@3c/port/endpoint/phandle=� 000000a5 | |-i2c@4c006000/camera@3c/port/endpoint/name=endpoint 656e64706f696e7400 | |-i2c@4c006000/camera@3c/port/name=port 706f727400 | |-i2c@4c006000/camera@3c/clock-names=xclk 78636c6b00 | |-i2c@4c006000/camera@3c/status=disabled 64697361626c656400 | |-i2c@4c006000/camera@3c/reset-gpios=/ 0000002f0000000400000001 | |-i2c@4c006000/camera@3c/phandle=� 000000a4 | |-i2c@4c006000/camera@3c/reg=< 0000003c | |-i2c@4c006000/camera@3c/name=camera 63616d65726100 | |-[mipi bridge:st,st-mipid02] | |-i2c@4c006000/stmipi@14/compatible=st,st-mipid02 73742c73742d6d69706964303200 | |-i2c@4c006000/stmipi@14/clocks=; 0000003b | |-i2c@4c006000/stmipi@14/VDDIN-supply=< 0000003c | |-i2c@4c006000/stmipi@14/clock-names=xclk 78636c6b00 | |-i2c@4c006000/stmipi@14/ports/#address-cells= 00000001 | |-i2c@4c006000/stmipi@14/ports/#size-cells= 00000000 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/hsync-active= 00000000 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/vsync-active= 00000000 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/remote-endpoint=> 0000003e | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/bus-width= 00000008 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/pclk-sample= 00000000 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/phandle=c 00000063 | |-i2c@4c006000/stmipi@14/ports/port@2/endpoint/name=endpoint 656e64706f696e7400 | |-i2c@4c006000/stmipi@14/ports/port@2/reg= 00000002 | |-i2c@4c006000/stmipi@14/ports/port@2/name=port 706f727400 | |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/remote-endpoint== 0000003d | |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/data-lanes= 0000000100000002 | |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/phandle=7 00000037 | |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/lane-polarities= 000000000000000000000000 | |-i2c@4c006000/stmipi@14/ports/port@0/endpoint/name=endpoint 656e64706f696e7400 | |-i2c@4c006000/stmipi@14/ports/port@0/reg= 00000000 | |-i2c@4c006000/stmipi@14/ports/port@0/name=port 706f727400 | |-i2c@4c006000/stmipi@14/ports/name=ports 706f72747300 | |-i2c@4c006000/stmipi@14/status=okay 6f6b617900 | |-i2c@4c006000/stmipi@14/reset-gpios=/ 0000002f0000000200000001 | |-i2c@4c006000/stmipi@14/VDDE-supply=< 0000003c | |-i2c@4c006000/stmipi@14/phandle=� 000000a6 | |-i2c@4c006000/stmipi@14/reg= 00000014 | |-i2c@4c006000/stmipi@14/name=stmipi 73746d69706900
5.1.2. Camera subsystem configuration debugging[edit source]
Refer to section #Get the topology of camera subsystem for basics on how to get and set camera subsystem configuration. Find below some debugging tips to help configuration handling.
5.1.2.1. subdevice configuration syntax error[edit source]
media-ctl utility returns an error when a configuration syntax mistake is made. Example here with erroneous entity name:
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" Unable to setup formats: Invalid argument (22)
In order to have details on the exact error, add -v option:
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" -v no such entity "dcmipp_dump_postpr" 'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480] ^
Another example with pad number mistake (only pad number 0 or 1 are allowed):
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" -v No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1 'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480] ^
Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8):
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" -v Invalid pixel code 'BA81' 'dcmipp_dump_postproc':0[fmt:BA81/640x480] ^
5.1.2.2. subdevice negotiation[edit source]
Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of configuration it can handle. media-ctl -v option when setting configuration shows the effective subdevice configuration.
Here is an example with 5 fps framerate not accepted by the subdevice and 30 fps selected instead:
media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/5 field:none]" -v Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0 Format set: RGB565_2X8_BE 640x480 Setting up frame interval 1/5 on pad gc2145 1-003c/0 Frame interval set: 1/30
Example with resolution negotiation (4000x4000 reduced to 2592x1944)
media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/4000x4000@1/15 field:none]" -v Setting up format RGB565_2X8_BE 4000x4000 on pad gc2145 1-003c/0 Format set: RGB565_2X8_BE 1600x1200
Example with format negotiation (AYUV8_1X32 not supported, defaults to RGB565_2X8_LE):
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:AYUV8_1X32/640x480]" -v Setting up format AYUV8_1X32 640x480 on pad dcmipp_dump_postproc/0 Format set: RGB565_2X8_LE
5.2. How to trace[edit source]
5.2.1. V4L2 userland API tracing[edit source]
Tracing of V4L2 userland API[3] can be enabled using command:
echo 0x3 > /sys/devices/platform/soc/5a000000.dcmipp/video4linux/video0/dev_debug
Traces are output in kernel log buffer:
dmesg [ 655.479016] video0: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_bytecap, bus=platform:dcmipp, version=0x00050f00, capabilities=0x852000011 [ 655.481591] video0: VIDIOC_G_FMT: type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesperl0 [ 655.481656] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, des' [ 655.481696] video0: VIDIOC_ENUM_FMT: index=1, type=vid-cap, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, des' [ 655.481732] video0: VIDIOC_ENUM_FMT: index=2, type=vid-cap, flags=0x0, pixelformat=YVYU little-endian (0x55595659), mbus_code=0x0000, des' [ 655.481767] video0: VIDIOC_ENUM_FMT: index=3, type=vid-cap, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, des' [ 655.481802] video0: VIDIOC_ENUM_FMT: index=4, type=vid-cap, flags=0x0, pixelformat=VYUY little-endian (0x59555956), mbus_code=0x0000, des' [ 655.481836] video0: VIDIOC_ENUM_FMT: index=5, type=vid-cap, flags=0x0, pixelformat=GREY little-endian (0x59455247), mbus_code=0x0000, des' [ 655.481873] video0: VIDIOC_ENUM_FMT: index=6, type=vid-cap, flags=0x0, pixelformat=BA81 little-endian (0x31384142), mbus_code=0x0000, des' [ 655.481909] video0: VIDIOC_ENUM_FMT: index=7, type=vid-cap, flags=0x0, pixelformat=GBRG little-endian (0x47524247), mbus_code=0x0000, des' [ 655.481944] video0: VIDIOC_ENUM_FMT: index=8, type=vid-cap, flags=0x0, pixelformat=GRBG little-endian (0x47425247), mbus_code=0x0000, des' [ 655.481981] video0: VIDIOC_ENUM_FMT: index=9, type=vid-cap, flags=0x0, pixelformat=RGGB little-endian (0x42474752), mbus_code=0x0000, des' [ 655.482017] video0: VIDIOC_ENUM_FMT: index=10, type=vid-cap, flags=0x1, pixelformat=JPEG little-endian (0x4745504a), mbus_code=0x0000, de' [ 655.482061] video0: VIDIOC_S_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0 [ 655.526124] video0: VIDIOC_REQBUFS: count=4, type=vid-cap, memory=mmap [ 655.526221] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4 [ 655.526272] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 655.526296] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4 [ 655.526339] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 655.526359] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=2, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4 [ 655.526401] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 655.526420] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=3, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4 [ 655.526462] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000 [ 655.527192] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0 [ 655.536625] video0: VIDIOC_STREAMON: type=vid-cap [ 655.536701] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0 [ 655.892309] video0: VIDIOC_STREAMOFF: type=vid-cap [ 655.892435] video0: VIDIOC_STREAMOFF: type=vid-cap [ 655.901481] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap
5.2.2. V4L2 core framework tracing[edit source]
Tracing of the V4L2 core framework[4] can be enabled using commands:
echo 0x3 > /sys/module/videobuf2_common/parameters/debug echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug
Traces are output in kernel log buffer:
dmesg [ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000 [ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000 [ 1218.656398] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 2, plane 0 offset 0x009a8000 [ 1218.669843] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 3, plane 0 offset 0x00e7c000 [ 1218.677503] videobuf2_common: [cap-7b50f8bf] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each [ 1218.688361] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 0, plane 0 successfully mapped [ 1218.696744] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 1, plane 0 successfully mapped [ 1218.705291] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 2, plane 0 successfully mapped [ 1218.712487] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 3, plane 0 successfully mapped [ 1218.722474] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 1218.730528] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 1218.738153] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded [ 1218.746012] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded [ 1218.756237] videobuf2_common: [cap-7b50f8bf] vb2_core_streamon: successful [ 1218.761969] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers [ 1218.822167] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer [ 1218.828383] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued [ 1218.838812] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded [ 1218.846549] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers [ 1218.855624] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer [ 1218.862118] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued [ 1218.872406] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded [ 1218.880029] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers [ 1218.889460] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer [ 1218.896756] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 2, state: dequeued [ 1218.905383] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded [ 1218.912309] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers [ 1218.923080] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer [ 1218.930392] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 3, state: dequeued [ 1218.944499] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded [ 1218.952210] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful [ 1218.959921] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful [ 1218.968800] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 0 [ 1218.978419] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 1 [ 1218.988204] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 2 [ 1218.997501] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 3
5.2.3. DCMIPP V4L2 kernel driver tracing[edit source]
DCMIPP dynamic debug traces[8] can be enabled using command:
echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control echo "module dcmipp_bytecap +p" > /sys/kernel/debug/dynamic_debug/control echo "module dcmipp_byteproc +p" > /sys/kernel/debug/dynamic_debug/control echo "module dcmipp_parallel +p" > /sys/kernel/debug/dynamic_debug/control
Here is an example with a 5Mp jpeg capture:
media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_BE/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_parallel':0[fmt:RGB565_2X8_LE/640x480]" media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"
v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw
dmesg [ 236.968937] dcmipp-parallel dcmipp-parallel.1.auto: dcmipp_parallel: format update: old:640x480 (0x1007, 8, 1, 2, 1) new:640x480 (0x1007, 3, 0, 0, 0) [ 236.987292] dcmipp-parallel dcmipp-parallel.1.auto: dcmipp_parallel: format update: old:640x480 (0x1007, 3, 0, 0, 0) new:640x480 (0x1008, 3, 0, 0, 0) [ 246.673178] dcmipp-bytecap dcmipp-bytecap.3.auto: dcmipp_dump_capture: format update: old:640x480 (0x50424752, 8, 1, 2, 0) new:640x480 (0x50424752, 8, 1, 2, 0) [ 246.673704] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup queue, count=4, size=614400 [ 246.691053] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [0] phy=0xd4a00000 size=614400 [ 246.691114] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [1] phy=0xd4b00000 size=614400 [ 246.691136] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [2] phy=0xd4c00000 size=614400 [ 246.691156] dcmipp-bytecap dcmipp-bytecap.3.auto: Setup [3] phy=0xd4d00000 size=614400 [ 246.691189] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [0] cdcc4929 phy=0xd4a00000 [ 246.691208] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [1] 8cfa694d phy=0xd4b00000 [ 246.691221] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [2] 57f4cf64 phy=0xd4c00000 [ 246.691233] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [3] 9a8d0e1c phy=0xd4d00000 [ 246.691397] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0FCTCR 0x00000003 [ 246.691415] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0FCTCR 0x00000000 [ 246.691425] dcmipp-byteproc dcmipp-byteproc.2.auto: SET DCMIPP_P0FCTCR 0x00000000 [ 246.691435] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0FCTCR 0x00000000 [ 246.691445] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0PPCR 0x00000180 [ 246.691455] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0PPCR 0x00000000 [ 246.691465] dcmipp-byteproc dcmipp-byteproc.2.auto: CLR DCMIPP_P0PPCR 0x00000400 [ 246.691474] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0PPCR 0x00000000 [ 246.691484] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0SCSTR 0x00000000 [ 246.691493] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0SCSZR 0x00000000 [ 246.691504] dcmipp-byteproc dcmipp-byteproc.2.auto: crop to 640x480 [ 246.691514] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0SCSTR 0x00000000 [ 246.691524] dcmipp-byteproc dcmipp-byteproc.2.auto: WR DCMIPP_P0SCSZR 0x81e00140 [ 246.691535] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_dump_postproc" is started [ 246.691547] dcmipp-parallel dcmipp-parallel.1.auto: WR DCMIPP_PRCR 0x02220000 [ 246.691557] dcmipp-parallel dcmipp-parallel.1.auto: SET DCMIPP_PRCR 0x00004000 [ 246.691567] dcmipp-parallel dcmipp-parallel.1.auto: WR DCMIPP_PRCR 0x02224000 [ 246.691576] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_parallel" is started [ 246.693135] dcmipp-bytecap dcmipp-bytecap.3.auto: "st-mipid02 1-0014" is started [ 246.832561] dcmipp-bytecap dcmipp-bytecap.3.auto: "gc2145 1-003c" is started [ 246.832598] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_P0FSCR 0x80000000 [ 246.832619] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0FSCR 0x80000000 [ 246.832635] dcmipp-bytecap dcmipp-bytecap.3.auto: Start with next [0] cdcc4929 phy=0xd4a00000 [ 246.832661] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0PPM0AR1 0xd4a00000 [ 246.832676] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [0] cdcc4929 phy=0xd4a00000 [ 246.832695] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0DCLMTR 0x80025800 [ 246.832709] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_P0FCTCR 0x00000008 [ 246.832724] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0FCTCR 0x00000008 [ 246.832738] dcmipp-bytecap dcmipp-bytecap.3.auto: SET DCMIPP_CMIER 0x00008600 [ 246.832753] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMIER 0x00008600 [ 247.094680] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR2 0x00000400 [ 247.094723] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMFCR 0x00000400 [ 247.094759] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0PPM0AR1 0xd4b00000 [ 247.094777] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [1] 8cfa694d phy=0xd4b00000 [ 247.128387] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR2 0x00000300 [ 247.128430] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMFCR 0x00000300 [ 247.128469] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_P0DCCNTR 0x00096000 [ 247.128498] dcmipp-bytecap dcmipp-bytecap.3.auto: Done [0] cdcc4929 phy=0xd4a00000 [ 247.129127] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR2 0x00000500 [ 247.129163] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMFCR 0x00000500 [ 247.129357] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0PPM0AR1 0xd4c00000 [ 247.129383] dcmipp-bytecap dcmipp-bytecap.3.auto: Write [2] 57f4cf64 phy=0xd4c00000 [ 247.149857] dcmipp-bytecap dcmipp-bytecap.3.auto: Queue [0] cdcc4929 phy=0xd4a00000 [ 247.150021] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_dump_postproc" is stopped [ 247.150040] dcmipp-parallel dcmipp-parallel.1.auto: CLR DCMIPP_PRCR 0x00004000 [ 247.150053] dcmipp-parallel dcmipp-parallel.1.auto: WR DCMIPP_PRCR 0x02220000 [ 247.150074] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR2 0x00000300 [ 247.150085] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMFCR 0x00000300 [ 247.150130] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_P0DCCNTR 0x0005f000 [ 247.150147] dcmipp-bytecap dcmipp-bytecap.3.auto: Done [1] 8cfa694d phy=0xd4b00000 [ 247.156441] dcmipp-bytecap dcmipp-bytecap.3.auto: "dcmipp_parallel" is stopped [ 247.156965] dcmipp-bytecap dcmipp-bytecap.3.auto: "st-mipid02 1-0014" is stopped [ 247.157010] dcmipp-bytecap dcmipp-bytecap.3.auto: "gc2145 1-003c" is stopped [ 247.157028] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_CMIER 0x00008600 [ 247.157043] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_CMIER 0x00000000 [ 247.157053] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_P0FCTCR 0x00000008 [ 247.157062] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0FCTCR 0x00000000 [ 247.157073] dcmipp-bytecap dcmipp-bytecap.3.auto: CLR DCMIPP_P0FSCR 0x80000000 [ 247.157083] dcmipp-bytecap dcmipp-bytecap.3.auto: WR DCMIPP_P0FSCR 0x00000000 [ 247.157100] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_PRSR 0x00030000 [ 247.157111] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_PRSR] =0x00030000 [ 247.157120] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_P0SR 0x00000007 [ 247.157129] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_P0SR] =0x00000007 [ 247.157139] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_P0DCCNTR 0x0005f000 [ 247.157148] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_P0DCCNTR]=0x0005f000 [ 247.157157] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR1 0x00000003 [ 247.157166] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_CMSR1] =0x00000003 [ 247.157175] dcmipp-bytecap dcmipp-bytecap.3.auto: RD DCMIPP_CMSR2 0x00000000 [ 247.157184] dcmipp-bytecap dcmipp-bytecap.3.auto: [DCMIPP_CMSR2] =0x00000000 [ 247.157198] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=0 (overrun=0, limit=0, nactive=0), underrun=0, vsync=2, frame=2, buffers=2, it=4
5.3. How to debug[edit source]
5.3.1. Errors[edit source]
Errors are unconditionally traced in kernel log:
dmesg [ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4
5.3.2. Memory tracking[edit source]
Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through DMA backend. Those allocations can be traced using:
echo "module dma_contiguous +p" > /sys/kernel/debug/dynamic_debug/control echo "module videobuf2_dma_contig +p" > /sys/kernel/debug/dynamic_debug/control
Here is the trace after a VGA YUV422 capture.
[11311.617688] vb2_dc_mmap: mapped dma addr 0xf1900000 at 0xb3b6a000, size 614400 [11311.617986] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xb3ad4000, size 614400 [11311.618071] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xb3a3e000, size 614400 [11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400
4 frames of VGA YUV422 frames: 640x480x2=614400 bytes
6. Source code location[edit source]
6.1. User space[edit source]
6.2. Kernel space[edit source]
- V4L2 core source code
- stm32-dcmipp V4L2 driver source code
- i2c camera sensor V4L2 drivers source code
7. References[edit source]
- ↑ Information about V4L2 Linux kernel framework on wikipedia.
- ↑ 2.0 2.1 Linux Media Infrastructure userspace API » Part IV - Media Controller API
- ↑ 3.0 3.1 3.2 3.3 Linux Media Infrastructure userspace API » Part I - Video for Linux API
- ↑ 4.0 4.1 Media subsystem kernel internal API » 1. Video4Linux devices
- ↑ MB1854 camera daughter board
- ↑ CN4 Camera sensor connector
- ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition
- ↑ How to use the kernel dynamic debug