How to stream RAW camera over network

Applicable for

STM32MP25x lines

---

1. Overview

This article explains how to stream camera content over network thanks to GStreamer application on top of V4L2 Linux^® kernel framework.

This article focuses on camera sensors that does not output compressed image or video content but raw content such as YUV, RGB, or Raw-bayer images.

Remote sending of images with a decent framerate requires compression to respect a reasonable bandwidth on the network. This could be achieved by compressing to JPEG image format or video formats like VP8 or H264.

Find below some examples of command lines allowing to capture a continuous stream of raw images then compress to JPEG, VP8 or H264 stream while playing it using various multimedia players, either local or remote.

2. Local streaming

Here is an example of a local preview loopback using gst-launch to capture raw pictures then compress to JPEG then decode and display.

ExpandFor STM32MP255 line , set up the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh #wait few seconds that image is correct then hit "CTRL+C" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! decodebin ! autovideosink

3. UDP streaming

An internet connection is required, for example, by plugging an Ethernet cable on the:

STM32MP257x-EV1 Evaluation board

CN17 Ethernet connector 1

ExpandFor STM32MP255 line , set up the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh <wait few seconds that image is correct then hit "CTRL+C"> media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

Get the IP address aa.bb.cc.dd of the host PC using the ifconfig command:

 ifconfig | grep "inet"
   inet addr:aa.bb.cc.dd  Bcast:10.201.23.255  Mask:255.255.252.0
   inet addr:127.0.0.1  Mask:255.0.0.0

Then fill the host= udpsink property with this IP address on the remote side to send UDP JPEG stream:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! rtpjpegpay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP JPEG stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG ! rtpjpegdepay ! jpegparse ! decodebin ! videoconvert ! autovideosink sync=false

A new window appears on the host PC displaying the camera content.

The same can be done with a lower bandwidth thanks to video compression. Here is an example with VP8:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! rtpvp8pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP VP8 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=VP8 ! rtpvp8depay ! decodebin ! videoconvert ! autovideosink sync=false

Another example with H264:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP H264 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=H264 ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Another H264 streaming example giving a 200 Kb/s bitrate constraint to limit network bandwidth:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,bitrate=200000" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000

Information

Refer to the V4L2_video_codec_overview#Controlling_encoder section for more details about video encoder available controls

open and royalty-free video compression format (see https://en.wikipedia.org/wiki/VP8) - NEW

Advanced Video Coding (AVC) or H.264 or MPEG-4 Part 10, is a video compression standard (see https://en.wikipedia.org/wiki/H264) - NEW