X-LINUX-AI - face recognition using TensorFlow Lite C++ API

This article explains how to get started on the TensorFlow Lite face recognition application.

1. Description

The face recognition application is capable of recognizing the face of a known (i.e. enrolled) user.

C/C++ TensorFlow Lite face recognition application

The application demonstrates a computer vision use case for face recognition where frames are grabbed from a camera input (/dev/videox) and compute by 2 neural network models (face detection and feace recognition) interpreted by the TensorFlow Lite framework.
A Gstreamer pipeline is used to stream camera frames (using v4l2src), to display a preview (using waylandsink) and to execute neural network inference (using appsink).
The result of the inference is displayed in the preview. The overlay is done using GtkWidget with cairo.
This combination is quite simple and efficient in terms of CPU overhead.

1.1. Camera capture

The camera frame capture has the following characteristics:

• Resolution is set to QVGA (320 x 240)
• Pixel color format is set to RGB565

1.2. Frame Preprocessing

The main preprocessing stage involved in the Face Reco application is the pixel color format conversion so to convert the RGB565 captured frame into a RGB888 frame.

1.3. Face Detection

The Face Detection block is in charge of finding the faces present in the input frame (QVGA, RGB888). In the current version of the application, the maximum number of faces that can be found is set to one. The output of this block is a frame of resolution 96 x 96 that contains the face found in the input captured frame.

1.4. Face Recognition

The Face Recognition block is in charge of extracting features and computing a signature (feature vector) corresponding to the input face.

1.5. Face Identification

The Face Identification block is in charge of computing the distance between:

• The vector produced by the Face Recognition block, and
• Each of the vectors stored in memory (and corresponding to the enrolled faces)

The output Face Identification block generates the two following outputs:

• a User Face ID corresponding to the minimum distance
• a similarity score

2. Installation

2.1. Install from the OpenSTLinux AI package repository

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

After having configured the AI OpenSTLinux package you can install X-LINUX-AI components for this application:

 apt-get install tflite-cv-apps-face-recognition-c++

Then restart the demo launcher:

 systemctl restart weston@root

3. How to use the application

3.1. Launching via the demo launcher

3.2. Executing with the command line

The label_tfl_gst_gtk C/C++ application is located in the userfs partition:

/usr/local/demo-ai/computer-vision/tflite-image-classification/bin/label_tfl_gst_gtk

It accepts the following input parameters:

 
Usage: ./bin/label_tfl_gst_gtk -m <model .tflite> -l <label .txt file>                                                                                            
                                                                                                                                                                   
-m --model_file <.tflite file path>:  .tflite model to be executed                                                                                                 
-l --label_file <label file path>:    name of file containing labels                                                                                               
-i --image <directory path>:          image directory with image to be classified                                                                                  
-v --video_device <n>:                video device (default /dev/video0)                                                                                           
--crop:                               if set, the nn input image is cropped (with the expected nn aspect ratio) before being resized,                              
                                      else the nn imput image is only resized to the nn input size (could cause picture deformation).                              
--frame_width  <val>:                 width of the camera frame (default is 640)                                                                                   
--frame_height <val>:                 height of the camera frame (default is 480)                                                                                  
--framerate <val>:                    framerate of the camera (default is 15fps)                                                                                   
--input_mean <val>:                   model input mean (default is 127.5)                                                                                          
--input_std  <val>:                   model input standard deviation (default is 127.5)                                                                            
--help:                               show this help  

• launch image classification based on the pictures located in /usr/local/demo-ai/computer-vision/models/mobilenet/testdata directory

 /usr/local/demo-ai/computer-vision/tflite-image-classification/bin/launch_bin_label_tfl_mobilenet_testdata.sh

Information

Note that you need to populate the testdata directory with your own data sets. The pictures are then randomly read from the testdata directory

Central processing unit

Artificial Intelligence