This article explains how to get started on the Face Recognition application running on STM32 microcontroller. The Face Recognition application is capable of recognizing the face of a know (i.e. enrolled) user.
This article provides an overview on the following topics:
- Overview of the required hardware setup
- Overview of the software architecture
- Description of the Face Reco application running on STM32
1. Hardware setup
The Face Reco application is running on a hardware setup made up of a STM32 microcontroller board connected to a camera module board.
1.1. STM32 board: STM32H747 Discovery Kit
The STM32H747I-DISCO is a complete demonstration and development platform for STMicroelectronics STM32H747XIH6 microcontroller, designed to simplify user application development. The STM32H747XIH6device is based on the high-performance Arm® Cortex®-M7 and Cortex®-M4 32-bit RISC cores. The Cortex®-M7 core operates at up to 480 MHz and the Cortex®-M4 core at up to 240 MHz.
The STM32H747XIH6device incorporates high-speed embedded memories with a dual-bank Flash memory of 2 Mbytes and of 1 Mbyte of RAM.
The other key specifications of the board are:
- On-board STLINK-V3E debugger/programmer
- USB OTG HS
- 4” capacitive touch LCD display module with MIPI® DSI interface
- 2 x 512-Mbit Quad-SPI NOR Flash memory
- 256-Mbit SDRAM
- 8-bit camera connector
- microSD™ card
1.2. Camera board
In the context of the Face Reco application the firmware supports the two following sensors:
- The OV5640 sensor
- The OV9655 sensor
(which is th eone mounted on the MB1379 camera daughterboard). Two camera module boards are supported:
1.2.1. STM32F4DIS-CAM camera module
The picture below shows the STM32F4DIS-CAM camera module featuring a OV9655 sensor:
1.2.2. B-CAMS-OMV based camera module
The picture below shows the B-CAMS-OMV mother board on which a MB1379 camera daughterboard is plugged:
The B-CAMS-OMV board is an adaptation which, on top of MB1379 camera daughterboard, enables to support camera board from OpenMV and Arducam.
In the context of the Face Reco application the firmware supports only the OV5640 sensor (which is the one mounted on the MB1379 camera daughterboard).
1.3. STM32H747I-DISCO connection to camera boards
The picture below shows how to connect the camera board to the STM32H747I-DISCO board using a flex cable:
2. Software architecture
The figure below depicts the software architecture of the Face recognition application:
Here is below a brief description of the middleware components:
2.1. STM32 AI Runtime
This is a STM32 optimized AI library generated by the X-CUBE-AI tool when generating the C neural network model.
2.2. STM32 Image
This is a library of functions for image preprocessing: rescaling, pixel format conversion etc.
2.3. STM32 Face Detect
Library containing functions for Face Detection purpose.
2.4. STM32 Face Reco
Library containing functions for Face Recognition purpose.
3. Description of Face Reco application
3.1. Frame processing flow
The figure below shows the different frame processing stages involved in the Face Recognition application:
3.1.1. Camera capture
The camera frame capture has the following characteristics:
- Resolution is set to QVGA (320x240)
- Pixel color format is set to RGB565
3.1.2. Frame Pre-processing
The main preprocessing stage involved in the Face Reco application is the pixel color format conversion so to convert the RGB565 captured frame into a RGB888 frame.
3.1.3. Face Detection
The Face Detection block is in charge of finding the faces present in the input frame (QVGA, RGB888). In the current version of the application, the maximum number of faces that can be found is set to one. The output of this block is a frame of resolution 96x96 that contains the face found in the input captured frame.
3.1.4. Face Recognition
The Face Recognition block is in charge of extracting features and computing a signature (feature vector) corresponding to the input face.
3.1.5. Face Identification
The Face Identification block is in charge of computing the distance between:
- The vector produced by the Face Recognition block, and
- Each of the vector stored in memory (and corresponding to the enrolled faces)
The output of the whole system is a user Face ID corresponding to the minimum distance.
3.2. Running the application
The application has two main operating modes:
3.2.1. The "nominal" mode
Operating mode during which the application is mapping the face contained in the input frame to a specific user Face ID.
3.2.2. The "enrollment" mode
Prior being able to link a face with a user Face ID, the system should have the user face ID recorded into is database. For that purpose, a user must enroll himself so to have his face's features (in the form of a feature vector) recorded into the memory.
It has to be noted that the current version of the Face Reco application does not support retention of the enrolled users. In other words, all the enrollment information are lost upon reset.
3.2.3. In practice
Practically, here is how it works:
3.2.3.1. Nominal mode
Upon reset, the system is running in the so-called nominal mode.
As soon as a face is detected within the camera captured frame, a rectangle box is drawn around it.
If the system is not able to match the detected face with one of the enrolled faces (either because the user face is not yet enrolled or because the Face Identification similarity score is lower than the default recognition threshold), the "No match" message is displayed on the right banner of the LCD screen and the rectangle box is drawn in red.
If the system is able to match the detected face with one of the enrolled faces, the rectangle box is drawn in green and a thumbnail (representing the user's enrolled face matching the detected face) is displayed on the right banner of the LCD screen along with the following information:
- The User Face ID
- The similarity score expressed in percentage (%).
3.2.3.2. Enrollment mode
I order to enroll his face, the user must press the blue joystick button present on the board. There exist two options for triggering the enrollment:
- Either the enrollment is triggered immediately and automatically following up a press on the joystick button (= default behavior upon reset)
- Or the enrollment is triggered after a 4 seconds countdown following up a press on the joystick button
In both cases, once the enrollment is completed the "ENROLLMENT COMPLETED RELEASE JOYSTICK" message is displayed and a thumbnail corresponding to the enrolled face is displayedvon the right banner of the LCD screen.
Run in internal memory
FPS@3.5
Default threshold so to have 1% FAR
Threshold configurable via "Change Settings" menu
etc..
basic enrollment with no retention