This article explains how to use a TensorFlow Lite[1], ONNX [2] or OpenVX [3] model in applications for image classification based on MobileNetV1 and MobileNetV2 neural network using the stai_mpu API on our STM32MPUs serie.
1. Description[edit source]
The image classification neural network model allows identification of the subject represented by an image. It classifies an image into various classes.
The application demonstrates a computer vision use case for image classification where frames are grabbed from a camera input (/dev/videox) and analyzed by a neural network model interpreted by OpenVX, TFLite or ONNX framework.
A Gstreamer pipeline is used to stream camera frames (using v4l2src), to display a preview (using gtkwaylandsink) and to execute neural network inference (using appsink).
The result of the inference is displayed in the preview. The overlay is done using GtkWidget with cairo.
This combination is quite simple and efficient in terms of CPU overhead.
The models used with this application are the MobileNet v1 downloaded from the TensorFlow Lite Hub[4] and the MobileNet v2 downloaded from the ST model zoo[5]
2. Installation[edit source]
2.1. Install from the OpenSTLinux AI package repository[edit source]
After having configured the AI OpenSTLinux package you can install X-LINUX-AI components for image classification application:
2.1.2. On STM32MP1x board[edit source]
The TFLite application will be installed with the XNNPACK delegate to accelerate the neural network inference on the CPU.
- To install this application, please use the following command:
x-linux-ai -i stai-mpu-image-classification-cpp-tflite
- Then, restart the demo launcher:
systemctl restart weston-graphical-session.service
2.2. Source code location[edit source]
- in the Openembedded OpenSTLinux Distribution with X-LINUX-AI Expansion Package:
- <Distribution Package installation directory>/layers/meta-st/meta-st-x-linux-ai/recipes-samples/image-classification/files/stai_mpu
- on GitHub:
2.3. Regenerate the package from OpenSTLinux Distribution (optional)[edit source]
Using the OpenSTLinux Distribution, you are able to rebuild the application.
- Set up the build environment:
cd <Distribution Package installation directory> source layers/meta-st/scripts/envsetup.sh
- Rebuild the application on STM32MP2x:
- Rebuild the application on STM32MP1x:
bitbake stai-mpu-image-classification-cpp-tflite -c compile
The generated binary is available here:
<Distribution Package installation directory>/<build directory>/tmp-glibc/work/cortexa35-ostl-linux/stai-mpu-image-classification-cpp/5.0.0-r0/stai-mpu-image-classification-cpp-5.0.0/stai_mpu