1. Article purpose[edit source]
This article describes how to run an inference on the STM32MPx using the STAI MPU Python API. It is an example based on an image classification application. The unified architecture of the API allows deploying the same application on all the STM32MPx platforms using several model formats.
 Information
|
| This article provides a simple inferencing example using the STAI MPU Python API. If you wish to explore more of the functions provided by the API, please refer to the STAI MPU Python Reference. |
2. STAI MPU Python API[edit source]
STAI MPU is a cross STM32MPx platforms machine learning and computer vision inferencing API with a flexible interface to run several deep learning models formats such as TFLite, ONNX and NBG. If you wish to learn more about the API structure please refer to STAI MPU: AI unified API for STM32MPUs. In the next section we explore, with a basic image-classification example, how to inference your models on the board using the STAI MPU Python API whether you are running a TFLite, an ONNX or an NBG model on either STM32MP1x or STM32MP2x.
3. Running an inference using the STAI MPU C++ API[edit source]
3.1. Install runtime prerequisites on the target[edit source]
After having configured the AI OpenSTLinux package, you can install the X-LINUX-AI components and the packages needed to run the example. First, we start by installing main packages needed for image processing which are Python Numpy, Python OpenCV.
Board $> apt-get install python3-numpy python3-opencv
Start by installing the python3-libstai module by running the following command:
x-linux-ai -i python3-libstai
Then, we will need to install the API plugins required during runtime depending on the model format used for the inference:
- If you are using a TFLite™ model, please run the following command:
x-linux-ai -i stai-mpu-tflite
- If you are using an ONNX™ model, please run the following command:
x-linux-ai -i stai-mpu-ort
- If you are running your model on an STM32MP2 board and running and NBG model, please run the following command:
x-linux-ai -i stai-mpu-ovx
| Information
|
| It is important to mention that the package stai-mpu-ovx is not available on STM32MP1x boards and the TFLite™ and ONNX™ runtimes supported by the API are running exclusively on CPU. |
3.2. Write a simple NN inference Python script[edit source]
from stai_mpu import stai_mpu_network
from numpy.typing import NDArray
from typing import Any, List
from pathlib import Path
from PIL import Image
from argparse import ArgumentParser
import cv2 as cv
import numpy as np
import time
def load_labels(filename):
with open(filename, 'r') as f:
return [line.strip() for line in f.readlines()]
if __name__ == '__main__':
parser = ArgumentParser()
parser.add_argument('-i','--image', help='image to be classified.')
parser.add_argument('-m','--model_file',help='model to be executed.')
parser.add_argument('-l','--label_file', help='name of labels file.')
parser.add_argument('--input_mean', default=127.5, help='input_mean')
parser.add_argument('--input_std', default=127.5,help='input std de')
args = parser.parse_args()
stai_model = stai_mpu_network(model_path=args.model_file)
# Read input tensor information
num_inputs = stai_model.get_num_inputs()
input_tensor_infos = stai_model.get_input_infos()
for i in range(0, num_inputs):
input_tensor_shape = input_tensor_infos[i].get_shape()
input_tensor_name = input_tensor_infos[i].get_name()
input_tensor_rank = input_tensor_infos[i].get_rank()
input_tensor_dtype = input_tensor_infos[i].get_dtype()
print("**Input node: 0 -Input_name:{} -Input_dims:{} -
input_type:{} -Input_shape:{}".format(input_tensor_name, input_tensor_rank,
input_tensor_dtype, input_tensor_shape))
# Read output tensor information
num_outputs = stai_model.get_num_outputs()
output_tensor_infos = stai_model.get_output_infos()
for i in range(0, num_outputs):
output_tensor_shape = output_tensor_infos[i].get_shape()
output_tensor_name = output_tensor_infos[i].get_name()
output_tensor_rank = output_tensor_infos[i].get_rank()
output_tensor_dtype = output_tensor_infos[i].get_dtype()
print("**Output node: 0 -Output_name:{} -Output_dims:{} -
Output_type:{} -Output_shape:{}".format(output_tensor_name,
output_tensor_rank, output_tensor_dtype, output_tensor_shape))
# Reading input image
input_width = input_tensor_shape[1]
input_height = input_tensor_shape[2]
input_image = Image.open(args.image).resize((input_width,input_height))
input_data = np.expand_dims(input_image, axis=0)
if input_tensor_dtype == np.float32:
input_data = (np.float32(input_data) - args.input_mean) /args.input_std
stai_model.set_input(index=0, input_data)
stai_model.run()
output_data = stai_model.get_output(index=0)
results = np.squeeze(output_data)
top_k = results.argsort()[-5:][::-1]
labels = load_labels(args.label_file)
for i in top_k:
if output_tensor_dtype == np.uint8:
print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))
else:
print('{:08.6f}: {}'.format(float(results[i]), labels[i]))
3.3. Download and prepare test data[edit source]
First create the directory to store test data:
mkdir stai_mpu_cpp_example
Next download the models and the test pictures:
wget -O stai_mpu_cpp_example/mobilenet_v2_1.0_224_int8_per_tensor.nb https://github.com/STMicroelectronics/meta-st-x-linux-ai/blob/main/recipes-samples/image-classification/models/files/mobilenet_v2_1.0_224_int8_per_tensor.nb wget -O stai_mpu_cpp_example/mobilenet_v2_1.0_224_int8_per_tensor.tflite https://github.com/STMicroelectronics/meta-st-x-linux-ai/blob/main/recipes-samples/image-classification/models/files/mobilenet_v2_1.0_224_int8_per_tensor.tflite wget -O stai_mpu_cpp_example/labels_imagenet_2012.txt https://github.com/STMicroelectronics/meta-st-x-linux-ai/blob/main/recipes-samples/image-classification/models/files/labels_imagenet_2012.txt wget -O stai_mpu_cpp_example/bird.jpg https://farm3.staticflickr.com/8008/7523974676_40bbeef7e3_o.jpg wget -O stai_mpu_cpp_example/plant.jpg https://c2.staticflickr.com/1/62/184682050_db90d84573_o.jpg
Once you have the data and the labels files needed for inferencing downloaded and ready, it is time to cross-compile you STAI MPU C++ based application.
3.4. Script and models deployment and launch of application[edit source]
Copy the script file and the test data directory containing your model onto the board:
scp -r stai_mpu_cpp_example/ root@<board_ip>:/path/ scp stai_mpu_img_cls root@<board_ip>:/path/
| Information
|
| The corresponding runtime plugin to your model should installed before running the binary. |
Connect to the board and launch the example:
./stai_mpu_img_cls stai_mpu_cpp_example/mobilenet_v2_1.0_224_int8_per_tensor.nb stai_mpu_cpp_example/bird.jpg
Running model: edgetpu_cpp_example/inat_bird_edgetpu.tflite and model: edgetpu_cpp_example/inat_plant_edgetpu.tflite for 2000 inferences [Bird image analysis] max value index: 659 value: 0.652344 [Plant image analysis] max value index: 1680 value: 0.964844 Using one Edge TPU, # inferences: 2000 costs: 106.278 seconds.
Where the max value index represents the index of the class detected and the value represents the confidence. On these particular pictures, the bird detected is a poecile atricapillus (black-capped chickadee) and the plant is a helianthus annuus (sunflower). The index and the name of each class are available in the labels_imagenet_2012.txt stored in the stai_mpu_cpp_example/bird.jpg directory.