X-LINUX-AI OpenSTLinux Expansion Package

X-LINUX-AI is an STM32 MPU OpenSTLinux Expansion Package that targets artificial intelligence for STM32MP1 Series devices.

It contains Linux AI frameworks, as well as application examples to get started with some basic use cases such as computer vision (CV).

It is composed of an OpenEmbedded meta layer, named meta-st-stm32mpu-ai, to be added on top of the STM32MP1 Distribution Package.

It brings a complete and coherent easy-to-build / install environment to take advantage of AI on the STM32MP1 Series.

1. Version v2.0.0

Information

This version has been validated against the OpenSTLinux ecosystem release v2.0.0 and validated on STM32MP157x-DKx and STM32MP157x-EV1 boards.

1.1. Contents

• TensorFlow Lite 2.2.0
• Google Edge TPU accelerator native support
• armNN 20.05
• OpenCV 4.1.x
• Python 3.8.x (enabling Pillow module)
• Support STM32MP15xF devices operating at up to 800MHz
• Python and C++ application samples
  • Image classification using TensorFlow Lite based on MobileNet v1 model
  • Object detection using TensorFlow Lite based on COCO SSD MobileNet v1 model
  • Image classification using Google Edge TPU based on MobileNet v1 model
  • Object detection using Google Edge TPU based on COCO SSD MobileNet v1 model
  • Image classification using armNN TensorFlow Lite parser based on MobileNet v1 model
  • Object detection using armNN TensorFlow Lite parser based on COCO SSD MobileNet v1 model

1.2. Validated hardware

As any software expansion package, the X-LINUX-AI is supported on all STM32MP1 Series but it has been validated on the following boards:

• STM32MP157C-DK2
• STM32MP157C-EV1
• STM32MP157A-EV1

1.3. Software structure

X-LINUX-AI-CV v2.0.0 Expansion Package Software structure

1.4. Install from the OpenSTLinux AI package repository

Information

STMicroelectronics packages repository service is provided for evaluation purpose only, its content could be updated at anytime without notice and therefore not approved for use in production.

All the generated X-LINUX-AI packages are available from the OpenSTLinux AI package repository service hosted at the non-browsable URL http://packages.openstlinux.st.com/AI.

This repository contains AI packages that could be simply installed using apt-* utilities, which are the same utilities used on a Debian system:

• the main group contains the selection of AI packages whose installation is automatically tested by STMicroelectronics
• the updates group is reserved for future use like package revision update.

You can install them individually or by group of packages.

1.4.1. Prerequisites

1. Flash the Starter Package on your SDCard
2. Your board has an internet connection either through the network cable or through a WiFi connection.

Information

If your internet access depends on a proxy server, you should define the http_proxy environment variable with the following command before any apt-* commands: export http_proxy='http://<proxy url>:<proxy port>/'

1.4.2. Configure the AI OpenSTLinux package repository

Once the board is booted, execute the following command in the console in order to configure the AI OpenSTLinux package repository:

 echo "deb http://extra.packages.openstlinux.st.com/AI/2.0 dunfell main updates" >> /etc/apt/sources.list.d/extra.packages.openstlinux.st.list
 echo "DPkg::Pre-Invoke {\"echo 'https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_licenses' ; echo ; \";};" >> /etc/apt/apt.conf.d/10-st-disclaimer-extra-AI
 echo "APT::Update::Pre-Invoke {\"echo 'https://wiki.st.com/stm32mpu/wiki/X-LINUX-AI_licenses' ; echo ;\";};" >> /etc/apt/apt.conf.d/10-st-disclaimer-extra-AI

1.4.3. Install AI packages

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

 apt-get update

1.4.3.1. Install all X-LINUX-AI packages

Package name	Command	Description
packagegroup-x-linux-ai	apt-get install packagegroup-x-linux-ai	Install all the X-LINUX-AI packages (TensorFlow Lite, Edge TPU, armNN, application samples and tools)

1.4.3.2. Install AI framework related packages

Package name	Command	Description
packagegroup-x-linux-ai-tflite	apt-get install packagegroup-x-linux-ai-tflite	Install X-LINUX-AI packages related to TensorFlow Lite framework (including application samples)
packagegroup-x-linux-ai-tflite-edgetpu	apt-get install packagegroup-x-linux-ai-tflite-edgetpu	Install X-LINUX-AI packages related to the Edge TPU framework (including application samples)
packagegroup-x-linux-ai-armnn-tflite	apt-get install packagegroup-x-linux-ai-armnn-tflite	Install X-LINUX-AI packages related to the armNN framework (including application samples)

1.4.3.3. Install individual packages

Package name	Command	Description
arm-compute-library	apt-get install arm-compute-library	Arm Compute Library (ACL)
arm-compute-library-tools	apt-get install arm-compute-library-tools	Arm Compute Library utilities (graph examples and benchmarks)
armnn	apt-get install armnn	arm Neural Network SDK (armNN)
armnn-tensorflow-lite	apt-get install armnn-tensorflow-lite	armNN TensorFlow Lite parser
armnn-tensorflow-lite-examples	apt-get install armnn-tensorflow-lite-examples	armNN TensorFlow Lite examples
armnn-tfl-benchmark	apt-get install armnn-tfl-benchmark	armNN benchmark application for TensorFlow Lite models
armnn-tfl-cv-apps-image-classification-c++	apt-get install armnn-tfl-cv-apps-image-classification-c++	armNN TensorFlow Lite C++ image classification example
armnn-tfl-cv-apps-object-detection-c++	apt-get install armnn-tfl-cv-apps-object-detection-c++	armNN TensorFlow Lite C++ object detection example
armnn-tools	apt-get install armnn-tools	armNN utilitites such as unitary tests
libflatbuffers1	apt-get install libflatbuffers1	Memory Efficient Serialization Library
python3-pillow	apt-get install python3-pillow	Python Imaging Library
python3-tensorflow-lite	apt-get install python3-tensorflow-lite	Python TensorFlow Lite inference engine
tensorflow-lite-tools	apt-get install tensorflow-lite-tools	Tensorflow Lite utilities
tflite-cv-apps-image-classification-c++	apt-get install tflite-cv-apps-image-classification-c++	TensorFlow Lite C++ image classification example
tflite-cv-apps-image-classification-python	apt-get install tflite-cv-apps-image-classification-python	TensorFlow Lite Python image classification example
tflite-cv-apps-object-detection-c++	apt-get install tflite-cv-apps-object-detection-c++	TensorFlow Lite C++ object detection example
tflite-cv-apps-object-detection-python	apt-get install tflite-cv-apps-object-detection-python	TensorFlow Lite Python object detection example
tflite-models-coco-ssd-mobilenetv1	apt-get install tflite-models-coco-ssd-mobilenetv1	TensorFlow Lite COCO SSD Mobilenetv1 model
tflite-models-mobilenetv1	apt-get install tflite-models-mobilenetv1	TensorFlow Lite Mobilenetv1 model

Information

If you need more information about how to use apt-* utilities check the Package repository for OpenSTLinux distribution article.

1.5. Re-generate X-LINUX-AI OpenSTLinux distribution

With the following procedure, you can re-generate the complete distribution enabling the X-LINUX-AI expansion package.
This procedure is mandatory if you want to update by yourself some frameworks or if you want to modify the application samples.
To know more, please expand the contents...

1.5.1. Download the STM32MP1 Distribution Package v2.0.0

Install the STM32MP1 Distribution Package v1.2.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).

1.5.1.1. Clone following git repositories into <Distribution Package installation directory>/layers/meta-st

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

 cd <Distribution Package installation directory>/layers/meta-st
 git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.0.0_dunfell

1.5.2. Set up the build environment

 cd ../..
 DISTRO=openstlinux-weston MACHINE=stm32mp1 source layers/meta-st/scripts/envsetup.sh

1.5.3. Add the new layers to the build system

 bitbake-layers add-layer ../layers/meta-st/meta-st-stm32mpu-ai

1.5.4. Build the image

 bitbake st-image-ai

Information

Note that building the image could take long time depending on the host computer performance.

1.5.5. Flash the built image

Follow this link to know how to flash the built image.

2. How to use the X-LINUX-AI Expansion Package

2.1. Material needed

To use the X-LINUX-AI-CV OpenSTLinux Expansion Package, choose one of the following materials:

• STM32MP157C-DK2 + an UVC USB WebCam
• STM32MP157C-EV1 with the built in camera
• STM32MP157A-EV1 with the built in camera

Optional:

• Google Edge TPU accelerator

2.2. Launch an AI application sample

Each application are described within a dedicated article that explain: the purpose, how to use, how to install and how to rebuild.

Application samples are available in the X-LINUX-AI application samples zoo page.

2.3. Enjoy running your own NN models

The above examples provide application samples to demonstrate how to execute NN Tensforflow Lite models easily on the STM32MP1.

You are free to update the C/C++ application or Python scripts for your own purposes, using your own NN Tensorflow Lite models.

Source code location are provided in application sample pages.

Artificial Intelligence

Microprocessor Unit

Linux^® is a registered trademark of Linus Torvalds.

Computer Vision

Flash memory shortened to gain space in titles, tables and block diagrams

Advanced Package Tool (see https://wiki.debian.org/Apt)

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Automatic current limit (LCD power improvement solution)

Software development kit (A programming package that enables a programmer to develop applications for a specific platform.)

USB Video Class

Neural Network