1. Description[edit source]
1.1. What is ST Edge AI?[edit source]
The ST Edge AI utility provides a complete and unified Command Line Interface (CLI) to generate, from a pre-trained Neural Network (NN), an optimized model or a library for all STM32 devices including MPU, MCU, ISPU and MSC. It consists on three main commands: analyze, generate and validate. Each command can be used, regardless of the other commands using the same set of common options (model files, output directory…), or any specific options.
In the case of the STM32MPU, you can use the generate command in the ST Edge AI to convert a Neural Network (NN) model to an optimized Network Binary Graph (NBG). This NBG is the only format that allows you to run an NN model using the STM32MP2x Neural Processing Unit (NPU) acceleration.
1.2. Main features[edit source]
ST Edge AI for STM32MPU is delivered as UNKNOWN FOR THE MOMENT, the tool is available only for Linux and already contains all the python environment required to run a conversion. The objective of this tool is to allow the user to convert and execute an NN model easily on the STM32MP2x platforms. For this, the tool allows the conversion of a quantized TensorFlow™ Lite[1] or ONNX™[2] model into a NBG format.
The Network binary graph (NBG) is the precompiled NN model format using the OpenVX™ graph representation. This is the only NN format that can be loaded and executed directly on the NPU of STM32MP2x boards.
The model provided to the tool must be quantized using the 8-bits per-tensor asymmetric quantization scheme to have the best performances. If the quantization scheme is 8-bits per-channel, the model mainly runs on GPU, instead of NPU.
The tool does not support the custom operators:
If the provided model contains custom operations, they will be automatically removed or the generation will fail.
If the output of the model is already post-processed using a custom operator such as TFLite post-process, the post-processing layer will be deleted. To avoid this situation issue, you can:
- Provide a model that does not include the custom post-process layer and code the post-process function inside your applications. The model will run on NPU and the post-process will be executed on CPU.
- Split your model to execute the core of model on the NPU using the NBG model and the post-processing layer on the CPU using your TensorFlow™ Lite or ONNX™ model with stai_mpu API LINK available soon.
Once this NBG is generated you can benchmark and/or develop your AI application, please check this articles presenting how to deploy your NN model and guide to benchmark your NN model.
2. Installation[edit source]
First, download the STEdgeAI tool: LINK
The ST Edge AI directory contains the main Python™ script to use the tool, a README file to explain how to use it, and several other directories like:
STEdgeAI
├── bin ├── build_scripts ├── deploy ├── get-pip.py ├── irs ├── lib ├── Package_license_stm32mpu.txt ├── passes ├── python ├── scripts ├── share ├── stedgeai ├── targets ├── uncompile -> stedgeai ├── uncompile.py └── utilities
TO BE MODIFIED WHEN DELIVERY PROCESS IS CLEAR ON STEDGEAI SIDE
usage: stedgeai --model FILE --target stm32|stellar-e|ispu|mlc [--workspace DIR] [--output DIR] [--no-report] [--no-workspace] [-h] [--version] [--tools-version] [--verbosity [{0,1,2,3}]] [--quiet] generate
3. How to use the tool[edit source]
3.1. Script utilization[edit source]
The main interface of the tool is stedgeai binary. To print the help that is specific to a target, use:
stedgeai --target stm32mp25 --help
All the features of the stedgeai binary for stm32mp25 target are as follow:
STEdgeAI Core v1.0.0-A1 (STM32MP2 1.0)
command:
generate must be the first argument (default='generate')
common arguments:
--model FILE, -m FILE
paths of the original model files
--target stm32|stellar-e|ispu|mlc
target/device selector
--workspace DIR, -w DIR
workspace folder to use (default: 'st_ai_ws')
--output DIR, -o DIR folder where generated files are saved (default: 'st_ai_output')
additional arguments:
--no-report do not generate report file
-h, --help show this help message and exit
--version print the version of the CLI
--tools-version print the versions of the third party packages
--verbosity [{0,1,2,3}], -v [{0,1,2,3}], --verbose [{0,1,2,3}]
set verbosity level
--quiet disable the progress-bar
3.2. Testing with example[edit source]
To generate a NBG model, you need to provide a quantized TensorFlow™ Lite or ONNX™ model and select the correct target: stm32mp25.
3.2.1. Generate NBG from TensorFlow™ Lite or ONNX™[edit source]
To convert a TensorFlow™ Lite model to NBG:
stedgeai generate -m path/to/tflite/model --target stm32mp25
To convert a ONNX™ model to NBG:
stedgeai generate -m path/to/onnx/model --target stm32mp25
This command generate two files: a .nb file which is the NBG model, a .txt file which is the report of the generate command.
- The .nb file is located in the output folder. If you use the option --output to specify a directory, the NBG is located inside, else the default output folder is st_ai_output.
- The report with the extension report_modelName_stm32mp25.txt file is located in the workspace folder. If you use the option –workspace to specify a directory, the report is located inside, else the default output folder is st_ai_ws.
3.2.2. Generate NBG from TensorFlow™ Lite or ONNX™ without report[edit source]
To generate a NBG model without the report simply add the --no-report option in your command line.
To convert a TensorFlow™ Lite model to NBG without the report:
stedgeai generate -m path/to/tflite/model --target stm32mp25 --no-report
To convert a ONNX™ model to NBG without the report:
stedgeai generate -m path/to/onnx/model --target stm32mp25 --no-report
This command generate a .nb file located in the output folder. If you use the option --output to specify a directory, the NBG is located inside, else the default output folder is st_ai_output.
4. References[edit source]