How to create a current sensing classifier using NanoEdge AI Studio

Revision as of 14:57, 6 April 2023 by Registered User (Emulator in NanoEdge™ AI Studio)

This article is a tutorial on how to use NanoEdge™ AI Studio to create a current sensing classifier running on the NUCLEO-U575ZI-Q development board together with an X-NUCLEO-IKA01A1 expansion board.

This tutorial shows:

  • how to program the Nucleo board for datalogging
  • how to use NanoEdge™ AI Studio to collect data, label data, find the best algorithm, run the NanoEdge AI Emulator (the Emulator in this article), and generate the library files for deployment
  • how to use STM32CubeIDE to modify codes, build the project, and program the board
  • and finally, how to run the classifier on the Nucleo board.

The main process of this tutorial is shown in the flowchart below:

Flowchart of this article


1 Prerequisites

1.1 Hardware

  • A NUCLEO-U575ZI-Q development board
  • An X-NUCLEO-IKA01A1 multifunctional expansion board
  • Two USB Type-A or USB Type-C® to Micro-B cables
  • A Windows® powered laptop or PC (Windows®10)
  • A multispeed USB powered fan with three speed modes: low, medium and high

Hardware prerequisite for current sensing classifier

1.2 Software

1.2.1 Source Code

  • A firmware package current-sensing-classifier-u5-wiki-package.zip is ready to download from www.st.com. Download the zip file here. The package includes:
  • Project source code in subfolder usb-current-sensing-classifier-u5
  • Two pre-compiled binary files in subfolder Binary:
  • usb-current-sensing-u5-datalog.bin for datalogging
  • usb-current-sensing-u5-detection.bin for detection
  • Library files generated by NanoEdge™ AI Studio in subfolder libneai_usb-current-sensing-classifier-u5.zip

1.2.2 NanoEdge™ AI Studio

  • NanoEdge™ AI Studio is a new Machine Learning (ML) technology to bring true innovation easily to the end-users. In just a few steps, developers can create an optimal ML library for their project, based on a minimal amount of data.
  • Download NanoEdge™ AI Studio for free and unlimited use on ST boards.
  • After registration, you will receive the license key in your registered email, which will be used to activate NanoEdge™ AI Studio when you launch it for the first time.
Info white.png Information
  • If you have any issues during installation or activation process, follow this Troubleshooting Guide to configure proxy or other settings.
  • If you need to run NanoEdge™ AI Studio on a virtual machine, submit a request on our Online Support platform.
  • To ensure your request is quickly routed to the correct support team, select Product / Application for Category, and NanoEdgeAIStudio for Product Category/Part Number.

1.2.3 STM32CubeIDE

  • STM32CubeIDE is an all-in-one multi-OS development tool, which is part of the STM32Cube software ecosystem. It is an advanced C/C++ development platform with peripheral configuration, code generation, code compilation, and debug features for STM32 microcontrollers and microprocessors.
  • Download STM32CubeIDE from st.com

1.2.4 Tera Term

  • Tera Term is an open-source and freely available software terminal emulator, which is used to display the classification results.
  • Users can download and install the latest version available from Tera Term.

2 Program NUCLEO-U575ZI-Q for datalogging

The first step is to connect the NUCLEO-U575ZI-Q to your PC and program it for datalogging mode with the provided binary file.

2.1 Board connection

Boards connection for current sensing classifier

The boards setup and cable connections are shown in the picture:

  1. Using the ARDUINO® Uno R3 connectors on X-NUCLEO-IKA01A1 extension board and the ST Zio connectors on NUCLEO-U575ZI-Q board, stack the two boards together. Make sure that the orientation is the same as depicted and that the extension board is fully inserted into the Nucleo board. There are some pins left open on CN9 and CN10 on the NUCLEO-U575ZI-Q board.
  2. Connect the Micro-B end of the USB cable to the Micro-B connector (J2) on X-NUCLEO-IKA01A1, then connect the USB Type-C® or Type-A end to an available USB port on your laptop or PC.
  3. Plug the USB fan into the USB Type-A connector (J1) on the X-NUCLEO-IKA01A1 extension board
  4. Connect the Micro-B end of the USB cable to the Micro-B connector (CN1) on the NUCLEO-U575ZI-Q, then connect the Type-A end to another available USB port on your laptop or PC.

At this stage, the board is ready to be programmed.

Warning white.png Warning
There is a possibility that the pins of the USB connector located at the bottom of the X-NUCLEO-IKA01A1 could come into contact with the jumper pins on the NUCLEO-U575ZI-Q. Take the necessary precautions to ensure that there is no physical contact between them, as it could result in damage to the boards.
Make sure no contact between two boards
Info white.png Information
For more information about the connectors, refer to UM2861 for NUCLEO-U575ZI-Q and UM1955 for X-NUCLEO-IKA01A1

2.2 Program NUCLEO-U575ZI-Q using a binary file

The easiest way to program the board is using the drag-and-drop method. Once you have the pre-built binary file in your local disk, the board can be very easily programmed with the binary by simply performing a drag-and-drop action:

Use drag-and-drop to program the NUCLEO-U575ZI-Q
  1. In Windows File Explorer, find the binary file ..\current-sensing-classifier-u5-wiki-package\Binary\usb-current-sensing-u5-datalog.bin
  2. Left click on the file and hold the left button on your mouse.
  3. Find the "NOD_U575ZI" disk in the Navigation pane on the left hand side, drag the binary to that disk and release the left button.
  4. When the transfer is done, the board is programmed successfully. The RED LED (LD3) blinks when running in datalogging mode.
Info white.png Information
If the drag-and-drop method does not work, there are two other alternative methods to program the board:

3 Create an n-class classification project in NanoEdge™ AI Studio

In this step, we will create an n-class classification project in NanoEdge™ AI Studio, and configure the project settings.

Open NanoEdge™ AI Studio on PC to see the welcome interface below. Then, follow the steps:

NanoEdge AI Studio welcome page
  1. Click on n-Class Classification to choose it as project type.
  2. Click on CREATE NEW PROJECT to create the project.

The Project settings page is displayed as below:

Project settings for current sensing classifier

  1. Give the project a name, for instance current-sensing-u5.
  2. Add a brief description for the project.
  3. Choose Current sensor 1 axis for Sensor type.
  4. Set a limit for the Library Max RAM, for instance 32 KB.
  5. Choose No Flash limit
  6. Click on TARGET SELECTION button to choose our target board. In the pop-up window:
  7. Target selection in project settings for current sensing classifier
    • Type U5 in the search bar to list all STM32U5 development boards.
    • Check the box for NUCLEO-U575ZI-Q to select it.
    • Click on CONFIRM botton to confirm and close this window.


    Save project settings and go to next step

  8. Finally, click on SAVE & NEXT to go to the next step.


4 Datalogging in NanoEdge™ AI Studio

In this step, we will alter the speed of the fan by pressing the button on the back, as well as applying clog to the fan to collect data for seven (7) classes: stop, low speed, medium speed, high speed, low speed clogged, medium speed clogged and high speed clogged.

First, we will use the programmed Nucleo board and NanoEdge™ AI Studio to collect the data for each class via serial (over USB) streaming. Go back to NanoEdge™ AI Studio, once the project is created, you see the window as shown below:

    Current sensing classifier - datalogging main window
  1. Click on ADD SIGNAL to open the "Import signal" window.

  2. Current sensing classifier - select USB for signal source
  3. In the pop-up window, choose FROM SERIAL (USB) as Type of signal source.

  4. Current sensing classifier - configure of datalogging via USB
  5. In the next window, select the correct Serial / COM port. Refresh if needed.
  6. Info white.png Information
    Current sensing classifier - how to find correct COM port with Windows Device Manage

    COM port number varies on different PCs. In this example, it is COM59. You can go to Windows Device Manager to look for STMicroelectronics STLink Virtual COM Port under Ports (COM & LPT)

  7. Select 115200 for Baudrate
  8. Check the box to define Max lines of 100 signals. By doing this, the data logger will automatically stop when 100 lines of signal are collected.
  9. Make sure that the fan is initially turned off. Then click on the START/STOP button to start datalogging. The real-time data and plotting will be shown.
  10. Datalogging for one class is finished for current sensing classifier
  11. Once 100 lines of signal are collected, the data logger will stop. Click on CONTINUE to move to the next step.

  12. Current sensing classifier - preview collected data and finish import for one class
  13. You can preview the collected data. If no problem, click on IMPORT to finish datalogging for this class.

  14. Current sensing classifier - rename a class


  15. Give a class name by typing in the Name box, then hit enter on keyboard or click on the check button to save the class name.
  16. Repeat step 1-9 to add all classes
    • Press the button on the back the fan to switch its speed between low, medium and high
    • Use the box of the fan to clog it (putting the box on the back of the fan) for clogged classes.
    • You finally have 7 classes in total, including stop, low speed, medium speed, high speed, low speed clogged, medium speed clogged and high speed clogged (You can have your own class names). The final result is shown below and you are ready for next step.
Current sensing classifier - finished datalogging for all classes


5 Benchmarking in NanoEdge™ AI Studio

In this step, we will use NanoEdge™ AI Studio to quickly and automatically find the optimized algorithm for our use case.

After collecting the data for all classes, click on "Benchmark" in NanoEdge™ AI Studio. You see the welcome interface as shown below:

    Initialize benchmark for current sensing classifier

  1. Click on RUN NEW BENCHMARK
  2. A "Select your Class" window will pop up. Select All classes and click on START.
  3. Select all classes for benchmarking
  4. Then the Studio starts the benchmarking, and you see the window below:
  5. Benchmarking in progress for current sensing classifier


    1. This area indicates the benchmark status:
      • How many benchmarks have run
      • Number of libraries tested
      • Elapsed time
    2. This area shows the progress of the running benchmark, and its time stamp
    3. This area shows the performance indicators. The higher balanced accuracy and confidence level are better, while the lower RAM and Flash sizes are better.
    4. This small log window displays notable information and events such as benchmark status, search speed per thread, new libraries found, or others
    5. This graph shows the evolution of the performance indicators over time, as thousands of candidate libraries are tested.
    6. This area contains control buttons:
      • You can pause the benchmark and resume later
      • You can stop the benchmark when the desired performance is reached
    Info white.png Information
    • Benchmarking can run for hours or even days, so you do not need to wait for it to completely finish. For demo purpose, a 90% score is good enough, which will normally be achieved within 5 minutes.
    • Regarding the performance indicators, you can refer to this NanoEdge™ AI Studio wiki for more details.

  6. Once you are satisfied with the performance during the library selection process, click on the "STOP" button to end the benchmarking. The optimized algorithm will be selected for deployment later. You see the details of this benchmark as below:
  7. Benchmark result for current sensing classifier

  8. Click on xx Libraries to open the window for more information on this benchmark, such as score, memory footprint and types for tested algorithms, as well as the confusion matrix of the selected algorithm
  9. Benchmark details for current sensing classifier


6 Emulator in NanoEdge™ AI Studio

In this step, we will use the Emulator in NanoEdge™ AI Studio to verify the selected library before we generate and deploy the library files.

After finishing the benchmark, click on "Emulator" in NanoEdge™ AI Studio to open the Emulator interface.

Initialize the emulator for current sensing classifier
  1. Click on the INITIALIZE EMULATOR button to open the Emulator

  2. Emulator results for current sensing classifier

  3. Click on FROM SERIAL (USB) as we did in datalogging step
  4. Select the same COM port as datalogging
  5. Select the same Baudrate as datalogging.
  6. Get the fan ready in any mode, and click on the "START/STOP" button. The real-time classification result is shown in the Live Repartition as blue bar.
    • Swtich between differnt speeds, and add clogs. Check the classification results.
    • Click on the "START/STOP" button to stop the emulation.
  • If the classification results are not accurate, you should go back to datalogging step to start over the datalogging and benchmarking process.
  • Otherwise you can go to the next step to generate the library files for deployment.
Info white.png Information
For more information about the Emulator, refer to this NanoEdge™ AI Studio wiki

7 Validation in NanoEdge™ AI Studio

The validation screen is used to summarize any selected benchmark or to compare the metrics of two benchmarks. After finishing testing in the Emulator, click on "Validation" in NanoEdge™ AI Studio to open the validation interface:

Validation step for current sensing classifier


  1. You can switch between the summary page and the comparison page.
  2. You can choose any completed benchmark to display its information
  3. You can export the validation report to pdf file for future reference
Info white.png Information
For more information about validation, refer to this NanoEdge™ AI Studio wiki

8 Deployment in NanoEdge™ AI Studio

In this step, we will generate the library files using NanoEdge™ AI Studio.

8.1 Generate library files in NanoEdge™ AI Studio

When you finish the Emulator, click on "Deployment" in NanoEdge™ AI Studio to open the deployment interface for library generation.

Deployment for current sensing classifier
  1. If you have multiple benchmarks, choose the one you want to compile. Otherwise, leave it as default.
  2. Check the box for Float abi to enable the hardware FPU operations.
  3. Click on the COMPILE LIBRARY button, and there will be a pop-up window as shown below:


Get library files for current sensing classifier
  • Click on the GET LIBRARY button, then there will be another pop-up window allowing you to choose a local directory to save the libraries:
Save NanoEdge library for current sensing classifier to local disk
  • Locate to a local directory you prefer, rename the file as you want, then click on Save.
  • Go to the directory you chose, make sure that the file libneai_*.zip is successfully saved there, extract the file, and you are ready for the next step.

8.2 Library files explanation

The NanoEdge AI static library for classification is the code that contains an AI model (such as a bundle of signal treatment, a Machine Learning model, or optimally tuned hyperparameters) designed to identify a sensor pattern in a class. All classes are defined by the user in NanoEdge™ AI Studio and are used during the training process of the AI model.

After the zip file is extracted, you have access to these folders and the files included:

NanoEdge AI Studio generated library files

There are three files we will need for next step: libneai.a, knowledge.h and NanoEdgeAI.h.

  • libneai.a is the C/C++ static library for the classifier.
  • knowledge.h is the header file that contains the knowledge buffer declaration.
  • NanoEdgeAI.h is the header file that contains function prototypes and sample declarations. An example of the code is shown below:

NanoEdge header files for current sensing classifier

Info white.png Information


9 Program NUCLEO-U575ZI-Q for Detection

In this step, we will add the library files into the project, import the project into STM32CubeIDE, modify the source code, then program the NUCLEO-U575ZI-Q for detection.

  1. Replace the old library files with the new ones
    • Go to the library folder, copy the file libneai.a
    • Go to the project folder ..\usb-current-sensing-classifier-u5\Core\Src, paste the file here, and click on Replace the file in the destination in the pop-up window.
    • Go back to the library folder, copy the files knowledge.h and NanoEdgeAI.h.
    • Go back to the project folder ..\usb-current-sensing-classifier-u5\Core\Inc, paste the files here, and click on Replace the files in the destination in the pop-up window.
  2. Import project into STM32CubeIDE workspace
    • Open STM32CubeIDE on your PC, create or choose a workspace, and click Launch to launch your workspace.
    • In the menu bar, click File -> Import...
    • In the pop-up window, select Existing Projects into Workspace under General, and click Next, as shown below:
    Pop-up window when importing project in STM32CubeIDE

    • In the next interface, click Browse..., find the project folder where you saved and then click Select Folder. Notice that there is a check mark in front of the detected project "usb-current-sensing-classifier-u5", then click Finish and wait for the project to be imported, as shown in the picture below:

    Import current sensing classifier project into STM32CubeIDE


  3. Build the project and program the NUCLEO-U575ZI-Q
    • In STM32CubeIDE, unfold the project in Project Explorer, find ..\Core\Src\main.c, and ..\Core\Inc\NanoEdgeAI.h. Double-click on the two files separately to open them in the editor, as shown below:
    STM32CubeIDE project explorer for current sensing classifier

    • In main.c, go to line 48 to find #define NEAI_MODE. Change the value to "1" if it is not, as shown below:
    NanoEdge project main.c line 48

    • Then go to NanoEdgeAI.h, find the buffer declarations in the commented part underneath, select the codes of "buffer for mapping class id to class name" const char *id2class[CLASS_NUMBER + 1]. Right click and choose "Copy" or simply press Ctrl+C on the keyboard to copy the codes:

    Copy buffer declaration from NanoEdgeAI.h for current sensing classifier


    • Go back to main.c, paste the copied codes below float output_class_buffer[CLASS_NUMBER]; to replace the existing declarations. As shown below:

    Paste copied code to main.c for current sensing classifier


    • Press CTRL + S to save the changes.
    • Make sure that the NUCLEO-U575ZI-Q board is connected to your PC.
    • Click on the "Run" button in the Toolbar to build and program the project:
    STM32CubeIDE Run button

    Info white.png Information
    • When running the project for the first time, an "Edit launch configuration properties" window will pop up. Keep all settings as default and click on OK to proceed.
    • If there is a pop-up window asking you to update to a newer firmware version of ST-LINK, click on Yes to open the upgrader. In the upgrader, click on Open in update mode and then click on Upgrade. Wait for the upgrade to complete, then close the upgrader and click RUN again in STM32CubeIDE.


    • You can see the log information in the console window. When the project is successfully built and programmed on the NUCLEO-U575ZI-Q, you see the following information:
    Download verified successfully
    
    Shutting down...
    Exit.
    
    • The GREEN LED(LD1) blinks when running in NEAI mode. Now you are ready for the final step - to run the classifier on NUCLEO-U575ZI-Q.

10 Run the Classifier on NUCLEO-U575ZI-Q

In this step, we will run the fan classifier on NUCLEO-U575ZI-Q and use Tera Term to display the classification results.

  • Once the Nucleo board is successfully programmed for detection, you can open Tera Term on your PC.
  • In Tera Term's menu bar, click on Setup -> Terminal.... In the New-line setting, change Receive to AUTO, and Transmit to CR:
Change the New-line setting in Tera Term


  • Go back to Tera Term's menu bar, click on Setup -> Serial port.... The serial port setup window will pop up as shown below:
Tera Term Serial port setup for STWIN fan classifier
  • Select the same COM port as we did in datalogging step for datalogging.
  • Select the Baudrate value "115200" for Speed.
  • Leave other settings as default, and click on New open.
  • Get the fan ready and make sure that all cables are firmly connected. The classification results will show in the interface, along with the probability of each class, as shown below:
Classification results in Tera Term for STWIN fan classifier

11 Resources