Difference between revisions of "AI:Introduction to Artificial Intelligence with STM32"

[quality revision]

Revision as of 10:50, 15 September 2020 (view source)

Vincent Abriou (talk | contribs)

m (→‎How to and Examples)

← Older edit

Latest revision as of 09:00, 6 April 2023 (view source)

Vincent Abriou (talk | contribs)

m

This page contains application examples, document references, tips and tricks and so on related to STM32 artificial intelligence.

1 What is AI at the edge[edit]

Smart objects are changing nearly every aspect of our daily lives, helping in our homes, workplaces, cities and factories. STMicroelectronics has been investing in AI for many years, recognizing that microcontrollers provide enough processing power to run AI software on smart objects and enable AI at the edge.

Local processing fixes many problems met when running AI algorithms purely in the Cloud:

Latency and high communication costs on connected devices
Reliability in case of limited network bandwidth or connectivity loss
Power efficiency on battery-operated devices, as high-bandwidth connectivity requires large device batteries, leading to a rise in cost
Data privacy since data is transformed locally and does not require to send all monitored data over the internet where security breach may occur during transmission or at Cloud storage

To simplify the development of AI algorithms on STM32 microcontrollers, STMicroelectronics has developed a solution called STM32Cube.AI.

2 STM32Cube.AI overview[edit]

STM32Cube.AI solution brings the following:

An STM32CubeMX extension called X-CUBE-AI to convert a Neural Network into optimized code for an STM32 microcontroller.
Software examples on STMicroelectronics development hardware boards are available for quick prototyping:
- Audio and motion sensing with the FP-AI-SENSING1 function pack working in combination of Bluetooth^® Low Energy smart sensor application and supported on the following development boards:
  - STM32 Nucleo-64 development board with STM32L476RG MCU NUCLEO-L476RG + Bluetooth^® Low Energy expansion board X-NUCLEOIDB05A1 + Motion MEMS and environmental sensor expansion board X-NUCLEO-IKS01A2
  - SensorTile development kit STEVALSTLKT01V1
  - SensorTile.box multisensor kit with portable sensor box STEVAL-MKSBOX1V1
  - STM32L4 Discovery kit IoT node B-L475E-IOT01A
- Vision on MCUs with the FP-AI-VISION1 function pack supported on the following development board:
  - STM32H747I-DISCO Discovery kit connected to the STM32F4DISB-CAM CAMS-OMVS camera daughterboard
- Vision AI on Linux for MPU with the X-LINUX-AI -CV expansion pack supported on the following development board:
  - Discovery kit with STM32MP157C MPU STM32MP157C-DK2 with a USB camera
  - Evaluation board with STM32MP157A MPU STM32MP157A-EV1
  - Evaluation board with STM32MP157C MPU STM32MP157C-EV1 with their built-in camera module
  - Avenger96 board from Arrow using Linaro™ 96Boards format based on the STM32MP157A microprocessor, either with a USB camera or the DesignCore® OV5640 camera mezzanine board from D3 Engineering
STM32 community with dedicated Neural Networks topic
AI Partner Program that brings expertise in Machine Learning and STM32 solutions

Check our list of resources for detailed information.

3 Videos related to STM32Cube.AI[edit]

4 STM32 supported by STM32Cube.AI[edit]

STM32Cube.AI supports all Arm^® Cortex^®-M4, Cortex^®-M33 and Cortex^®-M7-based MCUs, as well as STM32 MP1. Other MCUs are supported in our partner ecosystem or can be added on-demand.

STM32 family compatible with

STM32Cube.AI

and partners

STM32Cube.AI supports the following deep learning frameworks:

Keras^[1]
TensorFlow™^[2]
CaffeONNX^[3]
Lasagne^[4]
ONNX^[5]

Supported versions and layers for each framework are detailed in X-CUBE-AI release note. This documentation is available in X-CUBE-AI local installation folder at this location:

STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/5.0.0/Documentation/faqs.html

5 X-CUBE-AI overview[edit]

X-CUBE-AI is a tool provided to help users to:

Generate STM32 code from a Python™ Neural Network described in supported AI frameworks. It supports quantization scheme and optimizations for STM32, reducing memory requirement up to a factor of 4 and decreasing latency and power consumption up to a factor of 3.
Analyze the network and see if it fits on the target
Validate the generated C code on the PC or on the target against the original model

It is available through STM32CubeMX Graphical User Interface, or via Command Line Interface.

Software requirements

Supported operating systems and architectures

Windows^® 10 64-bit (x64)
Linux^® (tested on Red Hat^®, Fedora^® and Ubuntu^® 16.04) 64 bits
macOS^® (minimum version High Sierra)

Prerequisite

STM32CubeMX: 5.1.0 or higher is required

License

STM32CubeMX and X-CUBE-AI extension are delivered under the Mix Ultimate Liberty+OSS+3rd-party V1 software license agreement SLA0048.

6 STMicroelectronics Resources[edit]

All resources are gathered on STM32Cube.AI web page.

7 How to and Examples[edit]

Getting started

7.1 STM32 MCU[edit]

STM32Cube.AI
Expansion Package for Artificial Intelligence (AI)
How to integrate STM32Cube.AI generated code in OpenMV ecosystem
Getting started with FP-AI-NANOEDG1
NanoEdge AI Studio
Vision
Sensing

7.2 STM32 MPU[edit]

Getting started with X-LINUX-AI
- X-LINUX-AI OpenSTLinux Expansion Package
Vision

8 References[edit]

↑ Keras
↑ TensorFlow
↑ Lasagne

↑

↑

Caffe

ONNX

This page contains application examples, document references, tips and tricks and so on related to STM32 artificial intelligence.<br><br>

<div class="res-img">
[[File:STM32CubeAI banner.jpg|800px|center|link=https://www.st.com/stm32cubeai]]
</div>

==What is AI at the edge==
Smart objects are changing nearly every aspect of our daily lives, helping in our homes, workplaces, cities and factories. STMicroelectronics has been investing in AI for many years, recognizing that microcontrollers provide enough processing power to run AI software on smart objects and enable AI at the edge.<br>

{| class="st-table" style="text-align:left; margin: auto"
| style="border-style: hidden;" | <div class="res-img">[[File:EdgeAI.jpg|300px|link=]]</div>

| style="border-style: hidden; | Local processing fixes many problems met when running AI algorithms purely in the Cloud:
* Latency and high communication costs on connected devices
* Reliability in case of limited network bandwidth or connectivity loss
* Power efficiency on battery-operated devices, as high-bandwidth connectivity requires large device batteries, leading to a rise in cost
* Data privacy since data is transformed locally and does not require to send all monitored data over the internet where security breach may occur during transmission or at Cloud storage
|}

To simplify the development of AI algorithms on STM32 microcontrollers, STMicroelectronics has developed a solution called [https://www.st.com/stm32cubeai STM32Cube.AI].

==STM32Cube.AI overview==<div class="res-img">
[[File:STM32CubeAI.png|top|200px|none|link=https://www.st.com/stm32cubeai]] </div>
<br>

[https://www.st.com/stm32cubeai STM32Cube.AI] solution brings the following:
* An STM32CubeMX extension called [https://www.st.com/en/product/x-cube-ai X-CUBE-AI] to convert a Neural Network into optimized code for an STM32 microcontroller.
* Software examples on STMicroelectronics development hardware boards are available for quick prototyping:
**Audio and motion sensing with the [https://www.st.com/en/product/fp-ai-sensing1 FP-AI-SENSING1 function pack] working in combination of Bluetooth<sup>®</sup> Low Energy smart sensor application and supported on the following development boards:
*** STM32 Nucleo-64 development board with STM32L476RG MCU [https://www.st.com/en/product/nucleo-l476rg NUCLEO-L476RG] + Bluetooth<sup>®</sup> Low Energy expansion board [https://www.st.com/en/product/x-nucleo-idb05a1 X-NUCLEOIDB05A1] + Motion MEMS and environmental sensor expansion board [https://www.st.com/en/product/x-nucleo-iks01a2 X-NUCLEO-IKS01A2]
*** SensorTile development kit [https://www.st.com/en/product/steval-stlkt01v1 STEVALSTLKT01V1]
*** SensorTile.box multisensor kit with portable sensor box [https://www.st.com/en/product/steval-mksbox1v1 STEVAL-MKSBOX1V1]
*** STM32L4 Discovery kit IoT node [https://www.st.com/en/product/b-l475e-iot01a B-L475E-IOT01A]
**Vision on MCUs with the [https://www.st.com/en/product/fp-ai-vision1 FP-AI-VISION1 function pack] supported on the following development board:
*** [https://www.st.com/en/product/stm32h747i-disco STM32H747I-DISCO Discovery kit] connected to the STM32F4DIS-CAM camera daughterboard
**Vision[https://www.st.com/en/development-tools/b-cams-omv.html B-CAMS-OMVS camera daughterboard]
**AI on Linux for MPU with the [https://www.st.com/en/product/x-linux-ai-cv X-LINUX-AI-CV expansion pack] supported on the following development board:
*** Discovery kit with STM32MP157C MPU [https://www.st.com/en/product/stm32mp157c-dk2 STM32MP157C-DK2] with a USB camera
*** Evaluation board with STM32MP157A MPU [https://www.st.com/en/product/stm32mp157a-ev1 STM32MP157A-EV1]
*** Evaluation board with STM32MP157C MPU [https://www.st.com/en/product/stm32mp157c-ev1 STM32MP157C-EV1] with their built-in camera module
*** [https://www.arrow.com/en/products/avenger96/arrow-development-tools Avenger96 board] from Arrow using Linaro™ 96Boards format based on the [https://www.st.com/en/product/stm32mp157a STM32MP157A] microprocessor, either with a USB camera or the [https://www.arrow.com/en/reference-designs/d3-camera-mezz-ov5640-designcore-camera-mezzanine-board-with-5mp-micro-camera-module-enables-rapid-development-of-embedded-vision-applications/ccb2eb3402e6b1619e18f259c5f23651 DesignCore® OV5640 camera mezzanine board] from D3 Engineering
* [https://community.st.com/s/topic/0TO0X0000003iUqWAI/stm32-machine-learning-ai STM32 community] with dedicated Neural Networks topic
* AI [https://www.st.com/content/st_com/en/partner/partner-program.html Partner Program] that brings expertise in Machine Learning and STM32 solutions

Check our [https://www.st.com/content/st_com/en/stm32-ann.html#stm32-sann-resourcescontainer list of resources] for detailed information.

==Videos related to STM32Cube.AI==<div class="res-img">
[[File:AIS-Youtube.png|450px|center||link=https://www.youtube.com/playlist?list=PLnMKNibPkDnG9IC5Nl9vJg1CKMAO1kODW|AI solutions on STM32]]
</div>

==STM32 supported by STM32Cube.AI==

STM32Cube.AI supports all Arm<sup>®</sup> Cortex<sup>®</sup>-M4, Cortex<sup>®</sup>-M33 and Cortex<sup>®</sup>-M7-based MCUs, as well as STM32 MP1. Other MCUs are supported in our partner ecosystem or can be added on-demand.
<div class="res-img">
[[File:MCU support.png|frame|center|STM32 family compatible with STM32Cube.AI and partners]]
</div>

STM32Cube.AI supports the following deep learning frameworks:
* '''Keras'''<ref name=Keras_url>[https://keras.io Keras]</ref>

* '''TensorFlow™'''<ref name=TF_url>[https://www.tensorflow.org/lite TensorFlow]</ref>

* '''Caffe'''<ref name=Caffe_url>[https://caffe.berkeleyvision.org/ Caffe]</ref>

* '''Lasagne'''<ref name=Lasagne_url>[https://lasagne.readthedocs.io/en/latest/ Lasagne]</ref>

* '''ONNX'''<ref name=ONNX_url>[https://onnx.ai/ ONNX]</ref>


Supported versions and layers for each framework are detailed in X-CUBE-AI release note. This documentation is available in X-CUBE-AI local installation folder at this location: 
 STM32Cube/Repository/Packs/STMicroelectronics/X-CUBE-AI/5.0.0/Documentation/faqs.html

==X-CUBE-AI overview==
X-CUBE-AI is a tool provided to help users to:
* Generate STM32 code from a Python™ Neural Network described in supported AI frameworks. It supports quantization scheme and optimizations for STM32, reducing memory requirement up to a factor of 4 and decreasing latency and power consumption up to a factor of 3.
* Analyze the network and see if it fits on the target
* Validate the generated C code on the PC or on the target against the original model

It is available through [https://www.st.com/en/product/stm32cubemx STM32CubeMX] Graphical User Interface, or via Command Line Interface.

'''Software requirements'''

Supported operating systems and architectures
* Windows<sup>®</sup> 10 64-bit (x64)
* Linux<sup>®</sup> (tested on Red Hat<sup>®</sup>, Fedora<sup>®</sup> and Ubuntu<sup>®</sup> 16.04) 64 bits
* macOS<sup>®</sup> (minimum version High Sierra)

Prerequisite
* STM32CubeMX: 5.1.0 or higher is required

'''License'''

STM32CubeMX and X-CUBE-AI extension are delivered under the Mix Ultimate Liberty+OSS+3rd-party V1 software license agreement [https://www.st.com/sla0048 SLA0048].

==STMicroelectronics Resources==
All resources are gathered on [https://www.st.com/stm32cubeai STM32Cube.AI] web page.

==How to and Examples==*===STM32 MCU===
*'''STM32Cube.AI'''
**[https://www.st.com/content/ccc/resource/technical/document/user_manual/group1/69/bb/ec/5d/78/16/43/ce/DM00570145/files/DM00570145.pdf/jcr:content/translations/en.DM00570145.pdf Getting started X-CUBE-AI expansion package for artificial intelligence user manual]
**[[AI:X-CUBE-AI documentation | X-CUBE-AI documentation]]
**[[AI:X-CUBE-AI support of ONNX and TensorFlow quantized models | X-CUBE-AI support of ONNX and TensorFlow quantized models]]
**[[AI:STM32Cube.AI model performances | STM32Cube.AI model performances]]
**[[AI:How to automatize code generation and validation with X-CUBE-AI Expansion Package for Artificial Intelligence (AI)]
* [[How_to_integrate_STM32Cube.AI_generated_code_in_OpenMV_ecosystem | How to integrate STM32Cube.AI generated code in OpenMV ecosystem]]
* [[Getting_started_with_FP-AI-NANOEDG1 | Getting started with FP-AI-NANOEDG1]]

==CLI | How to automatize code generation and validation with X-CUBE-AI CLI]]
**[[AI:How to measure machine learning model power consumption with STM32Cube.AI generated application | How to measure machine learning model power consumption with STM32Cube.AI generated application]]
**[[AI:How to run larger models on STM32H747I-DISCO | How to run larger models on STM32H747I-DISCO]]
**[[AI:Deep Quantized Neural Network support | Deep Quantized Neural Network support]]
*'''NanoEdge AI Studio'''
**[[AI:NanoEdge_AI_Studio | NanoEdge AI Studio]]
**[[AI:NanoEdge_AI_Studio:_CLI | NanoEdge AI Studio: CLI]]
**[[AI:NanoEdge_AI_Library_for_anomaly_detection_(AD) | NanoEdge AI Library for anomaly detection (AD)]]
**[[AI:NanoEdge_AI_Emulator_for_anomaly_detection_(AD) | NanoEdge AI Emulator for anomaly detection (AD)]]
**[[AI:NanoEdge_AI_Library_for_n-class_classification_(nCC) | NanoEdge AI Library for n-class classification (nCC)]]
**[[AI:NanoEdge_AI_Emulator_for_n-class_classification_(nCC) | NanoEdge AI Emulator for n-class classification (nCC)]]
**[[AI:NanoEdge_AI_Library_for_1-class_classification_(1CC) | NanoEdge AI Library for 1-class classification (1CC)]]
**[[AI:NanoEdge_AI_Emulator_for_1-class_classification_(1CC) | NanoEdge AI Emulator for 1-class classification (1CC)]]
**[[AI:NanoEdge_AI_Library_for_extrapolation_(E) | NanoEdge AI Library for extrapolation (E)]]
**[[AI:NanoEdge_AI_Emulator_for_extrapolation_(E) | NanoEdge AI Emulator for extrapolation (E)]]
**[[AI:NanoEdge_AI_examples | NanoEdge AI examples]]
**[[AI:NanoEdge_AI_Anomaly_Detection_library_for_ISPU | NanoEdge AI Anomaly Detection library for ISPU]]
**[[AI:Datalogging guidelines for a successful NanoEdge AI project | Datalogging guidelines for a successful NanoEdge AI project]]
** [[AI:How_to_create_a_dual-tone_multi-frequency_classifier_using_NanoEdge_AI_Studio | How to create a dual-tone multi-frequency classifier using NanoEdge AI Studio]]
**[[AI:How to create a multi-state vibrations classifier using NanoEdge AI studio | How to create a multi-state vibrations classifier using NanoEdge AI studio]]
**[[AI:How to create a current sensing classifier using NanoEdge AI Studio | How to create a current sensing classifier using NanoEdge AI Studio]]
* '''Vision'''
** [[AI:FP-AI-FACEREC1 getting started | FP-AI-FACEREC1 getting started]]
** [[AI:How to use Teachable Machine to create an image classification application on STM32 | How to use Teachable Machine to create an image classification application on STM32]]
** [[AI:How to add AI model to OpenMV ecosystem | How to add AI model to OpenMV ecosystem]]
** [[AI:How to use transfer learning to perform image classification on STM32 | How to use transfer learning to perform image classification on STM32]]
* '''Sensing'''
** [[AI:FP-AI-MONITOR2 getting started | FP-AI-MONITOR2 getting started]]
** [[AI:FP-AI-MONITOR2 user manual | FP-AI-MONITOR2 user manual]]
** [[AI:FP-AI-MONITOR1 getting started | FP-AI-MONITOR1 getting started]]
** [[AI:FP-AI-MONITOR1 user manual | FP-AI-MONITOR1 user manual]]
** [[AI:FP-AI-MONITOR1 an introduction to the technology behind | FP-AI-MONITOR1 an introduction to the technology behind]]
** [[AI:FP-AI-MONITOR1 how to integrate a different AI Model for Human Activity Recognition (HAR) | FP-AI-MONITOR1 how to integrate a different AI Model for Human Activity Recognition (HAR)]]
** [[AI:FP-AI-NANOEDG1 V1.0 getting started | FP-AI-NANOEDG1 V1.0 getting started]]
** [[AI:FP-AI-NANOEDG1 V2.0 getting started | FP-AI-NANOEDG1 V2.0 getting started]]
** [[AI:FP-AI-NANOEDG1 V2.0 user manual | FP-AI-NANOEDG1 V2.0 user manual]]
** [[AI:How to perform anomaly detection using FP-AI-MONITOR1 | How to perform anomaly detection using FP-AI-MONITOR1]]
** [[AI:How to perform condition monitoring on STM32 | How to perform condition monitoring on STM32]]
** [[AI:How to perform motion sensing on STM32L4 IoTnode | How to perform motion sensing on STM32L4 IoTnode]]

===STM32 MPU===
*'''Getting started with X-LINUX-AI'''
** [https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_OpenSTLinux_Expansion_Package X-LINUX-AI OpenSTLinux Expansion Package]
*'''Vision'''
** [https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_-_face_recognition_using_TensorFlow_Lite_C%2B%2B_API X-LINUX-AI - face recognition]
** [https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_application_samples_zoo X-LINUX-AI - image classification]
** [https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_application_samples_zoo X-LINUX-AI - object detection]

==References==<references />

<noinclude>

[[Category:ArtificalArtificial Intelligence | 0]]
{{PublicationRequestId | 14747| 2020-01-29 | AnneJ ?}}</noinclude>

[Keras_url-1] Keras

[TF_url-2] TensorFlow

[Lasagne_url-4] Lasagne

[ONNX_url-3] ↑

[ONNX_url-3] ONNX

[1]

[2]

[3]

[4]

[5]

User

Wiki

Categories

General Books

Difference between revisions of "AI:Introduction to Artificial Intelligence with STM32"

Contents