X-LINUX-AI-CV OpenSTLinux expansion package

This message will disappear after all relevant tasks have been resolved.

Semantic MediaWiki

There are 1 incomplete or pending task to finish installation of Semantic MediaWiki. An administrator or user with sufficient rights can complete it. This should be done before adding new data to avoid inconsistencies.

This article provide a description of the content of the X-LINUX-AI-CV expansion package and explanation about how to use it.

1. Description[edit source]

X-LINUX-AI-CV is an STM32 MPU OpenSTLinux expansion package that targets artificial intelligence for computer vision.
This package contains AI and computer vision frameworks as well as application examples to start playing with some basic use cases.

STM32MPU Embedded Software with the X-LINUX-AI-CV OpenSTLinux expansion package

1.1. Current version[edit source]

X-LINUX-AI-CV v1.0.0

1.2. Contents[edit source]

Tensorflow Lite 2.0.0-beta1
OpenCV 3.4.x
Python 3.5.x (enabling Pillow module)
Python application examples
- Image classification example based on MobileNet v1 model
- Object detection example based on COCO SSD MobileNet v1 model

1.3. Software structure[edit source]

X-LINUX-AI-CV v1.0.0 Expansion Package Software structure

1.4. Supported hardware[edit source]

2. How to use it?[edit source]

2.1. Software installation[edit source]

Please refer to the STM32MP1 Artificial Intelligence expansion package article to build and install the X-LINUX-AI-CV software.

2.2. needed material[edit source]

To use the X-LINUX-AI-CV Expansion Package, you can use one of the following material:

STM32MP157C-DK2 + an UVC USB WebCam
STM32MP157C-EV1 with the built in camera
STM32MP157A-EV1 with the built in camera

3. AI application examples[edit source]

3.1. Python TensorFlowLite applications[edit source]

This part provides python applications examples based on TensorflowLite and OpenCV.
The applications integrate camera preview and test data picture that is then connected to the chosen TensorFlowLite model.

3.1.1. Image classification[edit source]

3.1.1.1. Description[edit source]

An image classification Neural Network model will allow you to identify what an image represents. It classifies an image into various classes.

Image classification application

The label_tfl_multiprocessing.py python script (located in the SDCard userfs partition: /usr/local/demo-ai/ai-cv/python/label_tfl_multiprocessing.py) is a multi-process python application for image classification.

The application enables OpenCV camera streaming (or test data pictures) and TensorFlowLite interpreter runing the NN inference based on the camera (or test data pictures) inputs.
The user interface is done thanks to python GTK.

3.1.1.2. How to use it[edit source]

The python scripts label_tfl_multiprocessing.py accept different input parameters:

-i, --image          image directory with images to be classified
-v, --video_device   video device (default /dev/video0)
--frame_width        width of the camera frame (default is 640)
--frame_height       height of the camera frame (default is 480)
--framerate          framerate of the camera (default is 30fps)
-m, --model_file     tflite model to be executed
-l, --label_file     name of file containing labels
--input_mean         input mean
--input_std          input standard deviation

3.1.1.3. Testing with MobileNet V1[edit source]

3.1.1.3.1. Default model: MobileNet V1 0.5 128 quant[edit source]

The default model used for tests is the mobilenet_v1_0.5_128_quant.tflite downloaded from tensorflow model

To ease the launch of the python script, two shell scripts are available:

launch image classification based on camera frame inputs

 /usr/local/demo-ai/ai-cv/python/launch_python_label_tfl_mobilenet.sh

launch image classification based on the picture located in /usr/local/demo-ai/ai-cv/models/mobilenet/testdata directory

 /usr/local/demo-ai/ai-cv/python/launch_python_label_tfl_mobilenet_testdata.sh

Information

Note that you need to populate the testdata directory with your own data sets.

The picture are then randomly read from the testdata directory

3.1.1.3.2. Testing another MobileNet v1 model[edit source]

You can test other model by downloading it directly on the STM32MP1 board. As example:

 curl http://download.tensorflow.org/models/mobilenet_v1_2018_02_22/mobilenet_v1_1.0_224_quant.tgz | tar xzv -C /usr/local/demo-ai/ai-cv/models/mobilenet/
 python3 /usr/local/demo-ai/ai-cv/python/label_tfl_multiprocessing.py -m /usr/local/demo-ai/ai-cv/models/mobilenet/mobilenet_v1_1.0_224_quant.tflite -l /usr/local/demo-ai/ai-cv/models/mobilenet/labels.txt -i /usr/local/demo-ai/ai-cv/models/mobilenet/testdata/

3.1.1.4. Testing with your own model[edit source]

The label_tfl_multiprocessing.py python script fits with Tensorflow Lite model format for image classification. Any model with a .tflite extension and a label file can be used with label_tfl_multiprocessing.py python script.
Feel free to update the label_tfl_multiprocessing.py python script to perfectly fit your needs.

3.1.2. Object detection[edit source]

3.1.2.1. Description[edit source]

An object detection Neural Network model will allow you to identify and locate a known object within an image.

Object detection application

The objdetect_tfl_multiprocessing.py python script (located in the SDCard userfs partition: /usr/local/demo-ai/ai-cv/python/objdetect_tfl_multiprocessing.py) is a multi-process python application for image classification.
The application enables OpenCV camera streaming (or test data pictures) and TensorFlowLite interpreter runing the NN inference based on the camera (or test data pictures) inputs.
The user interface is done thanks to python GTK.

3.1.2.2. How to use it[edit source]

The python scripts objdetect_tfl_multiprocessing.py accepts different input parameters:

-i, --image          image directory with images to be classified
-v, --video_device   video device (default /dev/video0)
--frame_width        width of the camera frame (default is 640)
--frame_height      height of the camera frame (default is 480)
--framerate          framerate of the camera (default is 30fps)
-m, --model_file     tflite model to be executed
-l, --label_file     name of file containing labels
--input_mean         input mean
--input_std          input standard deviation

3.1.2.3. Testing with Coco ssd MobileNet v1[edit source]

The model used for test is the detect.tflite downloaded from (https://www.tensorflow.org/lite/models/object_detection/overview Object detection overview]

To ease the launch of the python script, two shell scripts are available:

launch object detection based on camera frame inputs

 /usr/local/demo-ai/ai-cv/python/launch_python_objdetect_tfl_coco_ssd_mobilenet.sh

launch object detection based on the picture located in /usr/local/demo-ai/ai-cv/models/coco_ssd_mobilenet/testdata directory

 /usr/local/demo-ai/ai-cv/python/launch_python_objdetect_tfl_coco_ssd_mobilenet_testdata.sh

Information

Note that you need to populate the testdata directory with your own data sets.

The picture are then randomly read from the testdata directory

4. Enjoy running your own CNN[edit source]

The two above examples provides to you application samples to demonstrate how Tensforflow Lite CNN can be easily executed on the STM32MP1.

Feel free to get inspired by the python scripts in order to update them for your own purpose with your own CNN Tensorflow Lite models.