STM32Cube.AI model performances

This article provides performance results for a set of well-known or reference pre-trained Neural Network models.

Performance metrics verified by the MLCommons association have been published in the MLPerf™ Tiny v1.1 benchmark. Below are additional performance metrics measured by STMicroelectronics, which have not been verified by MLCommons [ST 1].

Info white.png Information
  • X-CUBE-AI[ST 2] is an expansion software for STM32CubeMX that generates optimized C code for STM32 microcontrollers and neural network inference. It is delivered under the Mix Ultimate Liberty+OSS+3rd-party V1 software license agreement[ST 3] with the additional component license schemes listed in the product data brief[ST 4]
  • The inference time, current and energy measurement process described is not done in a certified laboratory but can be reproduced by any user. The results are average values, which may vary depending on the input data (random data are currently used), the temperature, and the STM32 device itself.
  • Published data in this article is not contractual.
  • Copyright STMicroelectronics - All right reserved. Do not publish the following data without written consent of STMicroelectronics

1. Performance results

1.1. STM32 High Performance MCUs

STM32 High Performance MCUs inference time, memory footprint and energy at 3.3 V:

STM32 Board STM32
characteristics
Model
Source\Link
Flash
total
(Kbyte)
RAM
total
(Kbyte)
Proc
Time
(ms)
Cur.
(mA)
Energy
(mJ)
3.3 V
Version
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
FDMobileNet 0.25 224x224x3
quant tfl
[1]
188 Kbytes 166 Kbytes 43.3 ms 98 mA 14 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
MobileNet v2 128x128x3
quant tfl
520 Kbytes 257 Kbytes 72 ms 101 mA 24 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Object Detector SSD
MobileNet v1 0.25 192x192x3
544 Kbytes 298 Kbytes 118 ms 101 mA 39 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Yamnet 256
quant tfl
190 Kbytes 117 Kbytes 60 ms 103 mA 20.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
CNN2D_ST_HandPosture VL53L8CX 8 postures
27 Kbytes 3.8 Kbytes 0.14 ms 98 mA 0.05 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
HAR SVC
float, onnx
128 Kbytes 6.3 Kbytes 0.75 ms 82 mA 0.21 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 0.9 ms 90 mA 0.26 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 9.1 ms 100 mA 3 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Image Classif.
MLPerf™Tiny
123 Kbytes 49 Kbytes 26 ms 101 mA 8.6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H735 SMPS
STM32H735G-DK
Flash 1 Mbyte
RAM 564 Kbytes (432)
Freq 550 MHz
Visual Wake Word
MLPerf™Tiny
99 Kbytes 56 Kbytes 16 ms 99 mA 5 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
FDMobileNet 0.25 224x224x3
quant tfl
[2]
188 Kbytes 166 Kbytes 60 ms 66 mA 13 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
MobileNet v2 128x128x3
quant tfl
520 Kbytes 257 Kbytes 100 ms 67 mA 22 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Object Detector SSD
MobileNet v1 0.25 192x192x3
544 Kbytes 298 Kbytes 165 ms 66 mA 36 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Yamnet 256
quant tfl
190 Kbytes 117 Kbytes 86 ms 66 mA 18.7 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
CNN2D_ST_HandPosture VL53L8CX 8 postures
27 Kbytes 3.8 Kbytes 0.2 ms 64 mA 0.04 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
HAR SVC
float, onnx
128 Kbytes 6.3 Kbytes 1 ms 57 mA 0.19 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 1.22 ms 62 mA 0.24 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 12.7 ms 66 mA 2.7 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Image Classif.
MLPerf™Tiny
123 Kbytes 50 Kbytes 36 ms 66 mA 7.8 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H747 SMPS
STM32H747I-DISCO
Cortex®-M7
Flash 2 Mbytes
RAM 1 Mbyte (0.5)
Freq 400 MHz(1)
Visual Wake Word
MLPerf™Tiny
99 Kbytes 56 Kbytes 22 ms 66 mA 4.7 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
FDMobileNet 0.25 224x224x3
quant tfl
[3]
188 Kbytes 166 Kbytes 85 ms 43 mA 12.6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
MobileNet v2 128x128x3
quant tfl
520 Kbytes 257 Kbytes 140 ms 45 mA 21 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Object Detector SSD
MobileNet v1 0.25 192x192x3
544 Kbytes 298 Kbytes 232 ms 45 mA 34.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Yamnet 256
quant tfl
190 Kbytes 117 Kbytes 118 ms 45 mA 17.5 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
CNN2D_ST_HandPosture VL53L8CX 8 postures
27 Kbytes 3.8 Kbytes 0.28 ms 43 mA 0.04 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
HAR SVC
float, onnx
128 Kbytes 6.3 Kbytes 1.4 ms 38 mA 0.18 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 1.8 ms 40 mA 0.2 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 18.1 ms 44 mA 2.7 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Image Classif.
MLPerf™Tiny
124 Kbytes 49 Kbytes 52 ms 45 mA 7.7 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Visual Wake Word
MLPerf™Tiny
99 Kbytes 56 Kbytes 31 ms 45 mA 4.6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1

(1) On Cortex®-M7 core in SMPS mode 400 MHz instead of 480 max in LDO. The Cortex®-M4 is running on a while(1) infinite loop.

For a given STM32 in a fixed configuration, the current consumption is in the same range regardless of the model. it might however vary depending on the complexity and topology of the model. The following table is providing the average current consumption of the model listed in the table above table (excluding the Anomaly Detection model which has a specific topology). These data can be used as a first estimation of the current consumption and the energy consumption of a new model from just the measurement of its inference time. From the average inference time of t second and the average current of i Ampere for a given input voltage of u Volt. The average energy is easily computed as (t x i x u) in Joule.

STM32 Board STM32H735
550 MHz SMPS
STM32H747
400 MHz SMPS
STM32H7A3
280 MHz SMPS
Average
current (mA)
97 65 43


STM32Cube.AI (X-CUBE-AI) can also generate a TensorFlow™ Lite for Microcontroller (TFLm) runtime implementation (based on TensorFlow™ version 2.10 sha-1 = 79f6defor STM32Cube.AI v8.1.0). The following table is comparing the TFLm runtime to the X-CUBE-AI runtime, the Flash and RAM footprints include the code / runtime footprint on top of the weights and activation buffer.

STM32 Board STM32
characteristics
Model
Source/Link
Runtime Flash
(Kbyte)
RAM
(Kbyte)
Proc
Time
(ms)
Version
STM32H7A3 SMPS
NUCLEO-H7A3ZI-Q
Flash 2 Mbytes
RAM 1.4 Mbyte (1.18)
Freq 280 MHz
Image Classif.
MLPerf™Tiny
X-CUBE-AI 124 Kbytes 49 Kbytes 52 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
TFLm 160 Kbytes 55 Kbytes 98 ms TFLm sha-1 = 79f6de
STM32CubeIDE 1.12.1
Visual Wake Word
MLPerf™Tiny
X-CUBE-AI 99 Kbytes 56 Kbytes 31 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
TFLm 392 Kbytes 101 Kbytes 67 ms TFLm sha-1 = 79f6de
STM32CubeIDE 1.12.1

1.2. STM32 Ultra Low Power MCUs

STM32 Ultra Low Power MCUs inference time, memory footprint and energy at 3.3 V:

STM32 Board STM32
characteristics
Model
Source/Link
Flash
Total.
(Kbyte)
RAM
Total.
(Kbyte)
Proc
Time
(ms)
Cur.
(mA)
Energy
(mJ)
3.3 V
Version
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
FDMobileNet 0.25 224x224x3
quant tfl
[4]
187 Kbytes 166 Kbytes 209 ms 8.6 mA 6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
MobileNet v2 128x128x3
quant tfl
520 Kbytes 257 Kbytes 362 ms 9.6 mA 11.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Object Detector SSD
MobileNet v1 0.25 192x192x3
544 Kbytes 298 Kbytes 587 ms 9.6 mA 18.6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Yamnet 256
quant tfl
190 Kbytes 117 Kbytes 300 ms 9.7 mA 9.6 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
CNN2D_ST_HandPosture VL53L8CX 8 postures
24 Kbytes 3.8 Kbytes 0.66 ms 9.3 mA 0.02 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
HAR SVC
float, onnx
128 Kbytes 6.3 Kbytes 2.85 ms 9.6 mA 0.1 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 4.9 ms 9 mA 0.14 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 48.1 ms 9.6 mA 1.5 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Image Classif.
MLPerf™Tiny
123 Kbytes 50 Kbytes 134 ms 8.9 mA 3.8 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Visual Wake Word
MLPerf™Tiny
99 Kbytes 56 Kbytes 78.5 ms 9.7 mA 2.5 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
FDMobileNet 0.25 224x224x3
quant tfl
[5]
188 Kbytes 166 Kbytes 350 ms 24 mA 28 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
MobileNet v2 128x128x3
quant tfl
520 Kbytes 257 Kbytes 592 ms 24 mA 47 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Object Detector SSD
MobileNet v1 0.25 192x192x3
543 Kbytes 298 Kbytes 968 ms 24.4 mA 78 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Yamnet 256
quant tfl
188 Kbytes 117 Kbytes 488 ms 24.3 mA 39 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
CNN2D_ST_HandPosture VL53L8CX 8 postures
24 Kbytes 3.8 Kbytes 1.3 ms 23.6 mA 0.1 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
HAR SVC
float, onnx
126 Kbytes 6.3 Kbytes 4.6 ms 26 mA 0.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 6.8 ms 23 mA 0.51 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 81 ms 24 mA 6.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Image Classif.
MLPerf™Tiny
123 Kbytes 49 Kbytes 226 ms 24 mA 18 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32L4R5 LDO
NUCLEO-L4R5ZI
Flash 2 Mbytes
RAM 640 Kbytes
Freq 120 MHz
Visual Wake Word
MLPerf™Tiny
98 Kbytes 56 Kbytes 132 ms 24 mA 10.4 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32G474 LDO
NUCLEO-G474REI
Flash 512 Mbytes
RAM 128 Kbytes
Freq 170 MHz
Anomaly Detection
MLPerf™Tiny
280 Kbytes 6.39 Kbytes 5.15 ms 33 mA 0.56 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32G474 LDO
NUCLEO-G474REI
Flash 512 Mbytes
RAM 128 Kbytes
Freq 170 MHz
Key Word Spotting
MLPerf™Tiny
68 Kbytes 24 Kbytes 59 ms 35 mA 6.8 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32G474 LDO
NUCLEO-G474REI
Flash 512 Mbytes
RAM 128 Kbytes
Freq 170 MHz
Image Classif.
MLPerf™Tiny
123 Kbytes 49 Kbytes 161 ms 36 mA 19 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
STM32G474 LDO
NUCLEO-G474REI
Flash 512 Mbytes
RAM 128 Kbytes
Freq 170 MHz
Visual Wake Word
MLPerf™Tiny
99 Kbytes 56 Kbytes 95 ms 35 mA 11 mJ STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1

The following table is providing the average current consumption of the model listed in the table above table (excluding the Anomaly Detection model which has a specific topology). These data can be used as a first estimation of the current consumption and the energy consumption of a new model from just the measurement of its inference time. From the average inference time of t second and the average current of i Ampere for a given input voltage of u Volt. The average energy is easily computed as (t x i x u) in Joule

STM32 Board STM32U585
160 MHz SMPS
STM32L4R5
120 MHz LDO
Single Bank
STM32G474
170 MHz LDO
Average
current (mA)
9.3 24.1 27.2


STM32Cube.AI (X-CUBE-AI) can also generate a TensorFlow™ Lite for Microcontroller (TFLm) runtime implementation (based on TensorFlow™ version 2.10 sha-1 = 79f6de for STM32Cube.AI v8.1.0). The following table is comparing the TFLm runtime to the X-CUBE-AI runtime, the Flash and RAM footprints include the code / runtime footprint on top of the weights and activation buffer.

STM32 Board STM32
characteristics
Model
Source/Link
Runtime Flash
(Kbyte)
RAM
(Kbyte)
Proc
Time
(ms)
Version
STM32U585 SMPS
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Image Classif.
MLPerf™Tiny
X-CUBE-AI 123 Kbytes 49 Kbytes 134 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
TFLm 161 Kbytes 55 Kbytes 251 ms TFLm sha-1 = 79f6de
STM32CubeIDE 1.12.1
Visual Wake Word
MLPerf™Tiny
X-CUBE-AI 99 Kbytes 56 Kbytes 78.5 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
TFLm 393 Kbytes 101 Kbytes 176 ms TFLm sha-1 = 79f6de
STM32CubeIDE 1.12.1

2. SMPS vs LDO

Inference time, memory footprint and energy for SMPS and LDO power configuration at 3.3V :

STM32 Board STM32
characteristics
Model
Source/Link
PWR
config
Cur.
(mA)
Energy
(mJ)
Proc
Time
(ms)
Version
STM32U585
NUCLEO-U575ZI-Q
Flash 2 Mbytes
RAM 786 Kbytes
Freq 160 MHz
Image Classif.
MLPerf™Tiny
SMPS 8.9 3.8 134 STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
LDO 19.2 8.5 134 STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
Visual Wake Word
MLPerf™Tiny
SMPS 9.7 2.5 78.5 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1
LDO 19.4 5 78.5 ms STM32Cube.AI 8.1.0
STM32CubeIDE 1.12.1


3. Measurement process

On this performance only the Machine Learning model inference processing is reported. In a complete application, the sensor acquisition, the data conditioning and pre-processing must also be considered.

To give developers more control over their applications, ST introduced a new setting in STM32Cube.AI v8.1.0 to define priorities. If users choose the “Time” setting, the algorithm will take more RAM but have faster inference times. On the other hand, choosing “RAM” will have the smallest memory footprint and the slowest times. Finally, the default “Balanced” parameter finds the middle ground between the two approaches, providing a good compromise. For these measurements, the setting "balanced" is used.

The STM32 Board column indicates the STM32 reference and the board used for measurement. By default, the STM32 is configured in maximum performance configuration, so with maximum frequency and especially HCLK / AXI clock at maximal frequency. When a different setting is used it is specified (for instance lower frequency to use a different Voltage Scale or for STM32H7, lower HCLK/AXI frequency). Many STM32 embed a powerful switched-mode power supply (SMPS) that can be used to improve power efficiency when the supply voltage is high enough. When used instead of the integrated low-dropout regulator (LDO), power consumption is optimized by a factor equal to the ratio of the internal VCORE supply voltage to the VDD voltage. The improvement due to the SMPS depends only upon the SMPS efficiency and the VDD voltage. When SMPS is indicated it means that the internal voltage regulator used is the SMPS step-down converter instead of the LDO.

The STM32 Characteristics column provides the available internal Flash size, the full internal RAM size and the frequency. The RAM size includes the different kind of memories and banks, TCM, SRAM etc. For the time being, the buffers used by X-CUBE-AI must be placed in a continuous memory area, the maximal RAM size available in continuous area is provided between "()" if not equal to the full size. The frequency indicated is the operating frequency used for the test, so generally the maximal frequency. The only different case is with the STM32H747 Discovery kit (STM32H747I-DISCO), which is operating by default in SMPS power mode and therefore is limited to 400 MHz instead of 480 MHz. Data are rounded to 3 decimals.

The column Model Source/Links indicates the pre-trained ML model and the source, either how it was built / trained or where it can be downloaded. tfl stands for TensorFlow™ Lite .tflite model , h5 stands for Keras .h5 model, quant for quantized models on 8 bits.

The memory footprints are the one reported by X-CUBE-AI using the "Analyze" function (the version of X-CUBE-AI used is mentioned in the table).
The column Flash reports the Flash occupancy including the model weights, the runtime code generated by X-CUBE-AI to run the neural network and its constants (including the initialized tables).
The column RAM reports the RAM buffers occupancy, used to store the model activations as well as input and output buffers, and the RAM required by the runtime to inference the model. Note that to gain RAM space the "Use activation buffer for input buffer" and "Use activation buffer for the output buffer" options are selected (through X-CUBE-AI Advanced Settings panel).
For X-CUBE-AI runtime, the total Flash and RAM memory footprints are reported after an "Analyze" operation on the main panel by the fields Used Flash and Used RAM. The compiler used is gcc embedded in STM32CubeIDE.
For TensorFlow™ Lite for microcontroller runtime, the Flash and RAM memory footprints related to the runtime/code execution are computed from the memory map of the validation project of the given model built with STM32CubeIDE. The runtime/code part is computed taking into account all the modules used by tflite_micro. The STM32CubeIDE build options for TensorFlow™ Lite for microcontroller are the optimal ones (best compromise between speed and code size), -Ofast for GCC compiler and -Osize for G++ Compiler.

The column Proc Time reports the model inference processing time. When the current / energy is indicated, the measure is obtained through the X-CUBE-AI "System Performance" application following the process described on this WiKi article on power measurement. Otherwise the "Validation on target" application is used. In all cases, when generating the application, the selected clock source is always the HSI, X-CUBE-AI is generating first the optimal clock settings and eventually afterwards the clock is set to HSI. STM32CubeMX then autonomously reconfigures the clock settings.

Cur. and Energy is the current and energy computed following the process described in the WiKi article on power measurement. For STM32 Ultra Low Power microcontrollers, measurement is done with the X-NUCLEO-LPM01A power shield as described in the section 4.3.1 "Measure process when current is below 50 mA". For STM32 High Performance microcontrollers, measurement is done with the Qoitec Otii Arc power analyzer as described in the section 4.3.2 "Measure process when current is above 50 mA". In both cases, a 10 s window is used for averaging) and HSI is selected as the clock source.

Accuracy is not reported. X-CUBE-AI is not modifying the DL/ML model topology. The impact on accuracy should be limited. X-CUBE-AI is providing through the "Validation" application a way to measure the accuracy either on x86 or on the target. It can be used to check the eventual impact on accuracy. When running the "Validation on target" application several metrics are computed, one of them is the X-Cross providing error metrics between the original model executed in Python™ and the C model executed on the target. Random data can be used to compute the RMSE/MAE/L2R errors, however it is recommended to use true data to get the final accuracy. For more details on the metrics, refer to the X-CUBE-AI Embedded Documentation.

Note that accuracy check is important to compare a float model with a quantize model or when using the Weight compression feature of X-CUBE-AI for float models.

4. STMicroelectronics references

  1. MLPerf™ name and logo are trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.”
  2. X-CUBE-AI Expansion Package
  3. SLA0048 software license agreement
  4. DB3788 product data brief

See also: