X-LINUX-AI - image classification using TensorFlow Lite Python runtime

Applicable for STM32MP13x lines, STM32MP15x lines

This article explains how to experiment with TensorFlow Lite[1] applications for image classification based on the MobileNet v1 model using TensorFlow Lite Python runtime.

Info white.png Information
Python applications are good for prototyping but are less efficient than C/C++ application

1 Description[edit]

The image classification[2] neural network model allows identification of the subject represented by an image. It classifies an image into various classes.

Python TensorFlow Lite image classification application


The application enables three main features :

  • A camera streaming preview implemented using Gstreamer
  • A NN inference based on the camera (or test data pictures) inputs is being ran by the TensorFlow Lite[1] interpreter
  • A user interface implemented using Python GTK.

The performances depend on the number of CPUs available. The camera preview is limited to one CPU core where the TensorFlow Lite[1] interpreter is configured to use the maximum of the available resources.

The model used with this application is the MobileNet v1 downloaded from the Tensorflow Lite Hub[3].

2 Installation[edit]

2.1 Install from the OpenSTLinux AI package repository[edit]

Warning white.png Warning
The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

After having configured the AI OpenSTLinux package you can install the X-LINUX-AI components for this application:

 apt-get install tflite-cv-apps-image-classification-python

And restart the demo launcher:

- For OpenSTLinux distribution with a version lower than 4.0 use

 systemctl restart weston@root

- For other OpenSTLinux distribution use :

 systemctl restart weston-launch

2.2 Source code location[edit]

The label_tfl.py Python script is available:

  • in the Openembedded OpenSTLinux Distribution with X-LINUX-AI Expansion Package:
<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/tflite-cv-apps/files/image-classification/python/label_tfl.py
  • on the target:
/usr/local/demo-ai/computer-vision/tflite-image-classification/python/label_tfl.py
  • on GitHub:
https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/v2.2.0/recipes-samples/tflite-cv-apps/files/image-classification/python/label_tfl.py

3 How to use the application[edit]

3.1 Launching via the demo launcher[edit]

Demo launcher

3.2 Executing with the command line[edit]

The Python script label_tfl.py application is located in the userfs partition:

/usr/local/demo-ai/computer-vision/tflite-image-classification/python/label_tfl.py

It accepts the following input parameters:

usage: label_tfl.py [-h] [-i IMAGE] [-v VIDEO_DEVICE] [--frame_width FRAME_WIDTH]
                    [--frame_height FRAME_HEIGHT] [--framerate FRAMERATE] [-m MODEL_FILE]
                    [-l LABEL_FILE] [-e EXT_DELEGATE] [-p {std,max}] [--edgetpu]
                    [--input_mean INPUT_MEAN] [--input_std INPUT_STD] [--validation]
                    [--num_threads NUM_THREADS]

options:
  -h, --help            show this help message and exit
  -i IMAGE, --image IMAGE
                        image directory with image to be classified
  -v VIDEO_DEVICE, --video_device VIDEO_DEVICE
                        video device (default /dev/video0)
  --frame_width FRAME_WIDTH
                        width of the camera frame (default is 320)
  --frame_height FRAME_HEIGHT
                        height of the camera frame (default is 240)
  --framerate FRAMERATE
                        framerate of the camera (default is 15fps)
  -m MODEL_FILE, --model_file MODEL_FILE
                        .tflite model to be executed
  -l LABEL_FILE, --label_file LABEL_FILE
                        name of file containing labels
  -e EXT_DELEGATE, --ext_delegate EXT_DELEGATE
                        external_delegate_library path
  -p {std,max}, --perf {std,max}
                        [EdgeTPU ONLY] Select the performance of the Coral EdgeTPU
  --edgetpu             enable Coral EdgeTPU acceleration
  --input_mean INPUT_MEAN
                        input mean
  --input_std INPUT_STD
                        input standard deviation
  --validation          enable the validation mode
  --num_threads NUM_THREADS
                        Select the number of threads used by tflite interpreter to run inference
Info white.png Information
The --edgetpu and -p options are only available with a Coral Edge TPU[4] specific model. If you want to use the hardware acceleration please refer to the article provided for this purpose : X-LINUX-AI - image classification using Coral Edge TPU TensorFlow Lite Python runtime[5]

3.3 Testing with MobileNet V1[edit]

The model used for test is the mobilenet_v1_0.5_128_quant.tflite downloaded from Tensorflow Lite Hub[3]

Info white.png Information
The different objects the neural network is able to classify are listed in the labels.txt file located in the target:

/usr/local/demo-ai/computer-vision/models/mobilenet/labels.txt

To ease launching of the Python script, two shell scripts are available:

  • launch image classification based on camera frame inputs
 /usr/local/demo-ai/computer-vision/tflite-image-classification/python/launch_python_label_tfl_mobilenet.sh
  • launch image classification based on the pictures located in /usr/local/demo-ai/computer-vision/models/mobilenet/testdata directory:
 /usr/local/demo-ai/computer-vision/tflite-image-classification/python/launch_python_label_tfl_mobilenet_testdata.sh
Info white.png Information
Note that you need to populate the testdata directory with your own data sets.

The pictures are then randomly read from the testdata directory

4 References[edit]