How to stream RAW camera over network

Applicable for

STM32MP25x lines

---

1. Overview

This article will explain how to stream camera content over network thanks to GStreamer application on top of V4L2 Linux^® kernel framework.

This article focus on camera sensor which doesn't output compressed image or video content but raw content such as YUV, RGB or Raw-bayer images.

Remote sending of images with a decent framerate requires compression in order to respect a reasonable bandwidth on network. This could be achieved compressing to JPEG image format or video formats like VP8 or H264.

Find below some examples of command lines allowing to capture a continuous stream of raw images then compress to JPEG, VP8 or H264 stream while playing it using various multimedia players, either local or remote.

2. Local streaming

Here is an example of a local preview loopback using gst-launch to capture raw pictures then compress to JPEG then decode and display.

Warning

If Weston is configured with a "weston" user instead of a "root" user, please use the following command: su -l "weston" -c "your_weston_command" Note: Above tips may not be necessary on Yocto Project® versions greater than or equal to 4.2 ("Mickledore").

ExpandFor STM32MP255 line , setup the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh #wait few seconds that image is correct then hit "CTRL+C" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! decodebin ! autovideosink

3. UDP streaming

An internet connection is required, for example by plugging an ethernet cable on the:

STM32MP257x-EV1 Evaluation board

CN17 Ethernet connector 1

ExpandFor STM32MP255 line , setup the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh <wait few seconds that image is correct then hit "CTRL+C"> media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

Get the IP address aa.bb.cc.dd of the host PC using ifconfig command:

 ifconfig | grep "inet"
   inet addr:aa.bb.cc.dd  Bcast:10.201.23.255  Mask:255.255.252.0
   inet addr:127.0.0.1  Mask:255.0.0.0

Then fill the host= udpsink property with this IP address on the remote side in order to send UDP JPEG stream:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! rtpjpegpay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP JPEG stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG ! rtpjpegdepay ! jpegparse ! decodebin ! videoconvert ! autovideosink

A new window will popup on host PC displaying the camera content.

Same can be done with a lower bandwidth thanks to video compression. Here is an example with VP8:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! rtpvp8pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP VP8 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=VP8 ! rtpvp8depay ! decodebin ! videoconvert ! autovideosink

open and royalty-free video compression format (see https://en.wikipedia.org/wiki/VP8) - NEW

Advanced Video Coding (AVC) or H.264 or MPEG-4 Part 10, is a video compression standard (see https://en.wikipedia.org/wiki/H264) - NEW

The Linux Foundation® and Yocto Project® are registered trademarks of the Linux Foundation. Linux® is a registered trademark of Linux Torvalds