In this tutorial, we will use AI and a TOF to create a shifumi game. The goal is to demonstrate how you can use NanoEdge AI Studio and Arduino to be able to create any project you can think of using AI.
NanoEdge AI Studio is a tool developed by STMicroelectronics especially designed for embedded profiles to help them acquire an AI library to embed in their project, only using their data.
Through a simple and step by step process, you will collect data related to your use case, use the tool to get the best model with little effort
NanoEdge AI Studio libraries are compatible with any cortex M and since version 4.4, the tool is able to compile library ready to import directly in Arduino IDE.
1. Goals
- Learn how to use NanoEdge AI Studio to easily integrate AI in any future projects.
- Create a SHIFUMI game using TOF data and AI.
2. Hardware & Software Needed
Hardware:
- An Arduino GIGA R1 board
- An Arduino GiGA display shield
- An ST time of flight X-nucleo-53L5A1
- A Micro USB cable to connect the Arduino board to your desktop machine
Software:
- To program your board, you can use the Arduino Web Editor or install the Arduino IDE. We’ll give you more details on how to set these up in the following sections
- To create and get the AI model for the shifumi sign recognition, you need NanoEdge AI Studio v4.4 or above
note: For windows user, we recommend using Arduino IDE v1.8.19. Do not use the Microsoft store version.
Hardware setup:
Just a word concerning the setup montage. You need to plug the display screen on top of the GIGA R1 board and the TOF expansion board under the board like this:
Important:
We need to modify the I2C communication pins between the TOF and the Arduino board to avoid conflict with the display screen: Plug a jumper wire from the SDA1 pin to the 20SDA pin Plug another wire from the SCL1 pin to the 21SCL pin
3. NanoEdge AI Studio
For any part linked with NanoEdge AI Studio, you can take a look at the documentation for more information: https://wiki.st.com/stm32mcu/wiki/AI:NanoEdge_AI_Studio
3.1. Install:
First, we need to install NanoEdge AI Studio. After filling the form you will receive an email with your license. After launching the .exe file for the install, you will be asked to enter your license and you will be good with the install.
In case of troubles, here are useful links:
NanoEdge AI Studio wiki: license activation
Troubles shootings
3.2. Create a project:
4 kinds of projects are available in NanoEdge AI Studio:
- Anomaly detection (AD): to detect a nominal behavior and an abnormal one. Can be retrained directly on board
- 1 class classification (1c): Create a model to detect also a nominal behavior and an abnormal one but with only nominal data. (in case you cannot collect abnormal example)
- N class classification (Nc): Create a model to classify data in multiple classes that you define
- Extrapolation (Ex): Regression in short. To predict a value instead of a class from the input data (a speed or temperature for example)
In our case, we want to create an AI able to recognize three signs (scissor, paper and rock) so we click on N class classification >Create new project
In the project settings:
- Enter a project name
- Define a RAM and FLASH limit if you need
- Click SELECT TARGET, then go to the ARDUINO BOARD tab and select the GIGA R1 WiFi
- In sensor type, put GENERIC and 1 as the number of axis
- Then click SAVE AND NEXT
note: we are working with the TOF matrix of 64 values (8x8), but each measure are taking independently. That the reason why we use 1 axis.
3.3. Data collection:
In the Signal part of NanoEdge AI Studio, we need to import 4 datasets:
- Nothing: When we are not playing
- Paper
- Scissor
- Rock
Our TOF can collect data in a matrix of size 8x8, so every signal in our dataset if of size 64. We then need to collect data to create 4 datasets containing various example of one sign each.
Arduino code for data collection:
To collect data, create a new project in Arduino IDE and copy the following code:
printf("helloworld")
We need to add 2 libraries in the project:
- Wire.h for I2C communication: click on Sketch > Include Library > Wire
- SparkFun_VL53L5CX_Library: to use the TOF: Sketch > Include Library > Manage Library > SparkFun_VL53L5CX_Library and click install
Once ready, click on the verify sign to compile the code and then to the right arrow to flash the code on the board. Make sure that the board is connected to the pc beforehand.
Make sure that the right COM port is selected. Click on Tools > Port to select it. If the board is plugged, you should see its name displayed.
Back in NanoEdge:
In the step SIGNALS: Click on ADD SIGNAL Click on FROM SERIAL Zoom image.png Attachment options… image.png Zoom image.png Attachment options… image.png Make sure to select the right COM port Leave the Baudrate as it is Click on maximum number of lines and enter 500 Click START/STOP to log data Once finished click CONTINUE and then IMPORT
VERY IMPORTANT: Do not log data as if playing sifumi multiple times, log a sign continuously at different positions below the captor (up, down, left, right etc). For example, do the scissor sign and move below the captor while collecting the 500 signal without ever leaving the captor sight.
We really need to only have data corresponding to the class only. If you simulate playing to log data, you will have data corresponding to no signs (because you are not in the captor sight) and then data corresponding to the class.
Make also sure not to be to close to the TOF as it will not be able to see anything but a big object covering the whole matrix.