This article describes how to automate the generation and validation of machine learning model code with the X-CUBE-AI command-line interface (CLI). The example is provided for Windows® with the use batch script. It can be adapted easily to another operating system or through Python™. More information on CLI can be found in the embedded documentation.
1. Requirements and installations
- X-CUBE-AI latest version (latest tested v7.2)
- STM32CubeIDE latest version (latest tested v1.10.1)
- STM32CubeProgrammer (STM32CubeProg) latest version (latest tested v2.11.0)
- An STM32 evaluation tool; For this example, the NUCLEO-H723ZG Nucleo board is used.
- A model (Keras
.h5, TensorFlow™ Lite.tflite, or ONNX). For this example, we used a MobileNet v1 0.25 quantized with image inputs of 128x128x3.
The model, mobilenet_v1_0.25_128_quantized.tflite, is available for download.
Once X-CUBE-AI and STM32CubeIDE are installed, note the installation paths, which are usually the default ones on Windows® (replace "username" by your Windows® user account name and adapt to your tool version):
- For X-CUBE-AI: C:\Users\username\STM32Cube\Repository\Packs\STMicroelectronics\X-CUBE-AI\7.2.0
- For STM32CubeIDE: C:\ST\STM32CubeIDE_1.10.1\STM32CubeIDE\
- For arm gcc installed through STM32CubeIDE: C:\ST\STM32CubeIDE_1.10.1\STM32CubeIDE\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.10.3-2021.10.win32_1.0.0.202111181127\tools\bin
- For STM32CubeProgrammer: C:\Program Files\STMicroelectronics\STM32Cube\STM32CubeProgrammer\bin
2. Validation application generation
STM32CubeMX with the X-CUBE-AI plugin is used to generate the initial validation project for the targeted board.
2.1. Create a new project
Open STM32CubeMX and start the project using the board selector:
Select the board to use, NUCLEO-H723ZG in the example, and create a project without initializing all peripherals with their default mode:
2.2. Add X-CUBE-AI software pack
Select X-CUBE-AI software pack "Core" and "Validation" application:
Click on X-CUBE-AI software pack:
If, by default, the peripherals parameters are not set for the best performance, the system warns you. Select "Yes" to make sure that the maximal frequency is used.
X-CUBE-AI configures the default parameters to set the best performance. It configures also the UART used to report performance.
2.3. Generate the validation application
Upload a model, such as mobilenet_v1_0.25_128_quantized.tflite in the example.
In the example, the default memory options (kept from the version v7.2 of X-CUBE-AI) allocate input / output.
You can then generate the corresponding validation application as an STM32CubeIDE project in a dedicated directory, and then open it.
In STM32CubeIDE, build the project.
To check that the validation application works as expected, first program the board (either through STM32CubeIDE or STM32CubeProgrammer), then reset the board. Open the validation on the target panel and either select the board COM port or keep the automatic detection. Do not select the automatic compilation and download option as the board is already programmed with the right firmware.
Press "OK" and check that the validation runs correctly.
3. Automatic generation
The batch script below uses ST tools through the command-line interface to automatically:
- get the full memory footprint of a given model
- get the inference time
- get the validation metrics (validation done on random data)
In the script below, the local working directory is "C:\Project", which must correspond to the directory of the initial validation application project. There are two modes, defined by the variable "mode".
- The "analyze" mode, mode=analyze, does only an analysis to get the main memory footprint contributors (such as weights and activations buffers). In this mode, it is not needed to generate a validation application (section 2 can be skipped); Only X-CUBE-AI must be installed.
- The "validation" mode, mode=validate (default), does a validation to get the full memory footprint (weights, activations, and runtime), as well as inference time and validation metrics.
In the script, a temporary directory "tmp" is generated with two sub directories "stm32ai_ws" and "stm32ai_output". These directories are used by the X-CUBE-AI stm32ai CLI executable as workspace and output directory. The main outputs and reports are copied from these directories to a result directory. In the example "C:\Project\Results\STM32H723\mobilenet_v1_0.25_128_quant.tflite".
The total memory footprint for a given model must not consider only the size of the weights of the model (stored in flash memory) and the size of the activations, input, and output buffers. The size of the runtime code (flash memory and RAM) must also be considered.
The runtime memory footprint can only be measured by building the code with a toolchain.
A convenient way to measure the runtime footprint is to generate the code using the relocatable option.
A relocatable binary model designates a binary object that can be installed and executed anywhere in an STM32 memory sub-system. It contains a compiled version of the generated NN C-files, including the requested forward kernel functions and the weights. The principal objective is to provide a flexible way to upgrade an AI-based application without regenerating and programming the whole end-user firmware. This is the primary element to use, for example, the FOTA (Firmware Over-The-Air) technology. Refer to the "Relocatable binary model support" section of the embedded documentation for more information.
The relocatable binary corresponds to the model runtime code and the related report provides all the needed information to get the full memory footprint. Below is a report generated with the relocatable option:
Runtime memory layout (series="stm32h7")
------------------------------------------------------------------
section size (bytes)
------------------------------------------------------------------
header 100*
txt 33,716 network+kernel
rodata 628 network+kernel
data 14,884 network+kernel
bss 384 network+kernel
got 568*
rel 4,172*
weights 478,804 network
------------------------------------------------------------------
FLASH size 528,032 + 4,840* (+0.92%)
RAM size** 15,268 + 568* (+3.72%)
------------------------------------------------------------------
bin size 532,876 binary image
act. size 67,120 activations buffer
------------------------------------------------------------------
(*) extra bytes for relocatable support
(**) Full RAM = RAM + act. + IO if not allocated in activations buffer
The fields marked with (*) must not be considered as they relate to specific code of the relocatable feature and not to the runtime code itself. The full flash memory size is then composed of the "weights" size, here 478,804 bytes, and of the sections "txt", "rodata", and "data" amounting to 49,228 bytes for a total of 528,032 bytes. The full RAM size is then composed of the activations (here 67,120 bytes), input and output buffers (if not allocated in activations buffer), and the sections "data" and "bss", amounting to 15,268 bytes, for a total of 82,388 bytes. Note that to gain RAM space, the "Use activation buffer for input buffer" and "Use activation buffer for the output buffer" options are selected in this example and therefore do not add any memory footprint. Otherwise, they must be added to the total RAM size. Another way is to build the validation application and analyze the memory map.
To get the inference time, the script generates the code for the given network, and replaces the needed file in the initial validation application.
Then the validation application is rebuilt with STM32CubeIDE CLI.
The generated firmware .elf is loaded on the board, which must be connected to the PC through the ST-LINK USB and identified though a COM port.
Then the board is reset, and the validation application (with random data) is launched.
The validation report provides the model inference time as well as the validation error metrics.
The script copies the main information in the result .txt file:
STM32H723
mobilenet_v1_0.25_128_quant.tflite
weights (ro) : 478,804 B (467.58 KiB) (1 segment) / -1,391,568(-74.4%) vs float model
weights size : 478804 (1 segments)
activations (rw) : 67,120 B (65.55 KiB) (1 segment) *
activations size : 67120 (1 segments)
ram (total) : 67,120 B (65.55 KiB) = 67,120 + 0 + 0
FLASH size 528,032 + 4,840* (+0.92%)
RAM size** 15,268 + 568* (+3.72%)
(**) Full RAM = RAM + act. + IO if not allocated in activations buffer
duration : 45.774ms
cycles/MACC : 1.85
X-cross #1 n.a. 0.000772025 0.000057365 0.052570902 0.000007414 0.000772028 0.997128242 nl_30_0_conversion, ai_u8, (1,1,1,1001), m_id=[30]
Automatic code generation and validation with X-CUBE-AI CLI script:
@echo off
Rem ******************************************************************************
Rem Copyright (c) 2022 STMicroelectronics. All rights reserved.
Rem ******************************************************************************
Rem This software component is provided to you as part of a software package and
Rem applicable license terms are in the Package_license file. If you received this
Rem software component outside of a package or without applicable license terms,
Rem the terms of the BSD-3-Clause license shall apply.
Rem You may obtain a copy of the BSD-3-Clause at:
Rem https://opensource.org/licenses/BSD-3-Clause
Rem ******************************************************************************
Rem set the different paths to the tools and project location
Rem replace below username by your Windows user account name
set CUBE_FW_DIR=C:\Users\username\STM32Cube\Repository
set X_CUBE_AI_DIR=%CUBE_FW_DIR%\Packs\STMicroelectronics\X-CUBE-AI\7.2.0
set CUBE_IDE_DIR=C:\ST\STM32CubeIDE_1.10.1\STM32CubeIDE\
set ARM_GCC_DIR=C:\ST\STM32CubeIDE_1.10.1\STM32CubeIDE\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.10.3-2021.10.win32_1.0.0.202111181127\tools\bin
set CUBE_PROG_DIR=C:\Program Files\STMicroelectronics\STM32Cube\STM32CubeProgrammer\bin
set PATH=%X_CUBE_AI_DIR%\Utilities\windows;%CUBE_IDE_DIR%;%CUBE_PROG_DIR%;%ARM_GCC_DIR%;%PATH%
set locpath=C:\Project
Rem name: to identify the target STM32
set name=STM32H723
Rem STM32 series: stm32
set series=stm32h7
Rem model: model complete file name (including extension)
set model=mobilenet_v1_0.25_128_quant.tflite
Rem prjval: name of the initial X-CUBE-AI validation project
set prjval=STM32H723_project
Rem Com port is optional if only one board is connected to the PC
Rem set cport=COM18
Rem mode: analyze => do only an analyze to get main memory footprint contributors (ie weights and activations buffer)
Rem mode: validate => do a validation to get full memory footprint (weights, activations and runtime) as well as inference time
set mode=validate
Rem create directory
mkdir %locpath%\Results\%name%
mkdir %locpath%\Results\%name%\%model%
mkdir %locpath%\tmp
mkdir %locpath%\tmp\stm32ai_ws
mkdir %locpath%\tmp\stm32ai_output
set result_dir=%locpath%\Results\%name%\%model%
Rem write all result in a specific text file result.txt
del %result_dir%\result.txt
echo %name% >> %result_dir%\result.txt
echo %model% >> %result_dir%\result.txt
if %mode%==analyze (
Rem do only an analyze
@echo on
@echo Analyze %name% %model%
stm32ai analyze -m %locpath%\%model% --allocate-inputs --allocate-outputs -w %locpath%\tmp\stm32ai_ws -o %locpath%\tmp\stm32ai_output
copy /Y %locpath%\tmp\stm32ai_ws\network_report.json %result_dir%
copy /Y %locpath%\tmp\stm32ai_output\network_analyze_report.txt %result_dir%
findstr /b "weights" %result_dir%\network_analyze_report.txt >> %result_dir%\result.txt
findstr /b "activations" %result_dir%\network_analyze_report.txt >> %result_dir%\result.txt
findstr /b "ram" %result_dir%\network_analyze_report.txt >> %result_dir%\result.txt
@echo off
)
if %mode%==validate (
Rem do only a full validation to get full memory footprint (weights, activations and runtime) as well as inference time
@echo on
@echo Get the total memory footprint of the model %model% for %name%
stm32ai generate -m %locpath%\%model% --relocatable --allocate-inputs --allocate-outputs --series %series% -w %locpath%\tmp\stm32ai_ws -o %locpath%\tmp\stm32ai_output
copy /Y %locpath%\tmp\stm32ai_ws\network_report.json %result_dir%
copy /Y %locpath%\tmp\stm32ai_output\network_generate_report.txt %result_dir%\network_generate_relocatable_report.txt
findstr /b "weights" %result_dir%\network_generate_relocatable_report.txt >> %result_dir%\result.txt
findstr /b "activations" %result_dir%\network_generate_relocatable_report.txt >> %result_dir%\result.txt
findstr /b "ram" %result_dir%\network_generate_relocatable_report.txt >> %result_dir%\result.txt
findstr "FLASH" %result_dir%\network_generate_relocatable_report.txt >> %result_dir%\result.txt
findstr "RAM" %result_dir%\network_generate_relocatable_report.txt >> %result_dir%\result.txt
@echo Get the inference time of the model %model% for %name%
@echo Generate %name% %model%
stm32ai generate -m %locpath%\%model% --allocate-inputs --allocate-outputs --lib %X_CUBE_AI_DIR%\Middlewares\ST\AI -w %locpath%\tmp\stm32ai_ws -o %locpath%\tmp\stm32ai_output
copy /Y %locpath%\tmp\stm32ai_output\network*.c %locpath%\%prjval%\X-CUBE-AI\App\
copy /Y %locpath%\tmp\stm32ai_output\network*.h %locpath%\%prjval%\X-CUBE-AI\App\
@echo Building validation project %prjval%.elf
stm32cubeidec.exe --launcher.suppressErrors -nosplash -application org.eclipse.cdt.managedbuilder.core.headlessbuild -data workspace -import %locpath%\%prjval% -cleanBuild %locpath%\%prjval%\Debug
@echo Programming the board %locpath%\%prjval%\Debug\%prjval%.elf
STM32_Programmer_CLI.exe -c port=SWD mode=UR -w %locpath%\%prjval%\Debug\%prjval%.elf
Rem If the .bin file is preferred, the command-line must be adapted to add the start address
Rem STM32_Programmer_CLI.exe -c port=SWD mode=UR -w %locpath%\%prjval%\Debug\%prjval%.bin 0x08000000
STM32_Programmer_CLI.exe -c port=SWD -hardRst
TIMEOUT 1
@echo Launch the validation application
Rem stm32ai validate -m %locpath%\%model% -q %modelquant% --allocate-inputs --allocate-outputs --mode stm32 -d %cport% -w %locpath%\tmp\stm32ai_ws -o %locpath%\tmp\stm32ai_output
stm32ai validate -m %model% --allocate-inputs --allocate-outputs --mode stm32 -w %locpath%\tmp\stm32ai_ws -o %locpath%\tmp\stm32ai_output
copy /Y %locpath%\tmp\stm32ai_output\network_validate_report.txt %result_dir%
findstr "duration" %result_dir%\network_validate_report.txt >> %result_dir%\result.txt
findstr "cycles/MACC" %result_dir%\\network_validate_report.txt >> %result_dir%\result.txt
findstr "X-cross" %result_dir%\\network_validate_report.txt >> %result_dir%\result.txt
@echo off
)
PAUSE
Copyright (c) 2022 STMicroelectronics. All rights reserved.