X-LINUX-AI - image classification using armNN TensorFlow Lite parser

This article explains how to experiment armNN applications for image classification based on MobileNet v1 model using the TensorFLow Lite parser

1. Description

The image classification neural network model allows identification of the subject represented by an image. It classifies an image into various classes.

C/C++ armNN image classification application using TensorFlow Lite parser

The application demonstrate a computer vision use case for image classification where frames are grabbed from a camera input (/dev/videox) and analyzed by a neural network model interpreted by the armNN framework.
Gstreamer pipeline is used to stream camera frames (using v4l2src), to display a preview (using waylandsink) and to execute neural network inference (using appsink).
The result of the inference is displayed on the preview. The overlay is done using GTK widget with cairo.
This combination is quite simple and efficient in term of CPU consumption.

The model used with this application is the MobileNet v1 (floating point model) downloaded from the Tensorflow Lite hosted models.

2. Installation

2.1. Install from the OpenSTLinux AI package repository

After having configured the AI OpenSTLinux package you can install X-LINUX-AI components for this application:

 apt-get install armnn-tfl-cv-apps-image-classification-c++

And restart the demo launcher:

 systemctl restart weston@root

2.2. Source code location

• in the Openembedded OpenSTLinux distribution with X-LINUX-AI expansion package:

<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/armnn-cv-apps/files/tfl-image-classification/src

• on GitHub:

https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/dunfell/recipes-samples/armnn-cv-apps/files/tfl-image-classification/src

2.3. Re-generate the package from OpenSTLinux distribution (optional)

Using the Openembedded OpenSTLinux distribution, you are able to rebuild the application.

• Before doing so, you need to install the X-LINUX-AI expansion package.
  
• Then you can execute the following bitbake comand to rebuild the application:

 bitbake armnn-tfl-cv-apps-image-classification-c++ -c compile

• The generated binary is available here:

<Distribution Package installation directory>/<build directory>/tmp-glibc/work/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi/armnn-tfl-cv-apps-image-classification-c++/1.0-r0/armnn-tfl-cv-apps-image-classification-c++-1.0/tfl-image-classification/src

3. How to use it

3.1. Launch application via the demo launcher

3.2. Or execute with command line

The label_armnn_tfl_gst_gtk C/C++ application is located in the userfs partition:

/usr/local/demo-ai/computer-vision/armnn-tfl-image-classification/bin/label_armnn_tfl_gst_gtk

It accepts the following input parameters:

 
Usage: ./label_armnn_tfl_gst_gtk -m <model .tflite> -l <label .txt file>                                                              
                                                                                                                                      
-m --model_file <.tflite file path>:  .tflite model to be executed                                                                    
-l --label_file <label file path>:    name of file containing labels                                                                  
-b --backend <device>:                preferred backend device to run layers on by default. Possible choices: CpuAcc, CpuRef          
-i --image <directory path>:          image directory with image to be classified                                                     
-v --video_device <n>:                video device (default /dev/video0)                                                              
--crop:                               if set, the nn input image is cropped (with the expected nn aspect ratio) before being resized, 
                                      else the nn imput image is only resized to the nn input size (could cause picture deformation). 
--frame_width  <val>:                 width of the camera frame (default is 640)                                                      
--frame_height <val>:                 height of the camera frame (default is 480)                                                     
--framerate <val>:                    framerate of the camera (default is 15fps)                                                      
--input_mean <val>:                   model input mean (default is 127.5)                                                             
--input_std  <val>:                   model input standard deviation (default is 127.5)                                               
--help:                               show this help 

3.3. Testing with MobileNet V1

The model used for test is the mobilenet_v1_0.5_128.tflite downloaded from Tensorflow Lite hosted models

To ease launching of the application, two shell scripts are available:

• launch image classification based on camera frame inputs

 /usr/local/demo-ai/computer-vision/armnn-tfl-image-classification/bin/launch_bin_label_armnn_tfl_mobilenet.sh

• launch image classification based on the pictures located in /usr/local/demo-ai/computer-vision/models/mobilenet/testdata directory

 /usr/local/demo-ai/computer-vision/armnn-tfl-image-classification/bin/launch_bin_label_armnn_tfl_mobilenet_testdata.sh

Information

Note that you need to populate the testdata directory with your own data sets. The pictures are then randomly read from the testdata directory

Central processing unit

Artificial Intelligence