How to perform anomaly detection using FP-AI-MONITOR1

This message will disappear after all relevant tasks have been resolved.
Semantic MediaWiki
There are 1 incomplete or pending task to finish installation of Semantic MediaWiki. An administrator or user with sufficient rights can complete it. This should be done before adding new data to avoid inconsistencies.
Under construction.png The delivery of this wiki page is under construciton

This article aims to provide a step-by-step guide to set up a vibration-based condition monitoring solution for a USB fan using FP-AI-MONITOR1 and an STEVAL-STWINKT1B board. The anomaly detection AI libraries to be used in this tutorial will be generated using NanoEdgeTM AI Studio and the software used to program the sensor board is provided as a function pack that can be downloaded from the ST website.

In this article you learn:

  • using high-speed datalogger binary to log the vibration data on microSD card on STEVAL-STWINKT1B,
  • generating the AI library for condition monitoring using NanoEdgeTM AI Studio,
  • integrating the generated libraries to FP-AI-MONITOR1, and programming the sensor-node for condition monitoring, and
  • performing the condition monitoring on the USB fan setup using live vibration data.

1. Requirements

To follow and reproduce the steps provided in this article the reader requires the following parts.

1.1. Hardware

To perform all the steps in this document, users need following hardware components.

  • an STEVAL-STWINKT1B development kit board,
  • a windows® powered laptop/PC (Windows® 7, 8, or 10),
  • an STLINK-V3MINI,
  • Two microUSB cables, one to connect the sensor-board to the PC, and another one for the STLINK-V3MINI,
  • a microSDTM card formatted as FAT32-FS for the data logging operation,
  • a USB MicroSDTM card reader to read the data from the MicroSDTM card to the computer,
  • 2 x Micro USB cables to connect the motor control board, STLINK-V3Mini and STWIN,
  • a USB powered fan that can be ordered from this link
how to AD hardware requirements.png

1.2. Software

  • One of the following three IDEs
  • Tera Term
    Tera Term is an open-source and freely available software terminal emulator, which is used to host the CLI of the FP-AI-MONITOR1 through a serial connection. Users can download and install the latest version available from Tera Term.
  • NanoEdgeTM AI Studio 2.1.*
    The anomaly detection libraries used in this demo are generated by NanoEdge AI Studio. For this, the user needs to generate a project in the Studio and get the libraries from it. These libraries then can be embedded in the FP-AI-MONITOR1. The more information on the studio can be found on st.com.
  • Python 3.7.X
    To process the datalogs and to prepare the data in the form which is accepted by the NanoEdge AI Studio, Python utility scripts are provided. These scripts are available under path /FP-AI-MONITOR1_V1.0.0/Utilities/AI_Resources/NanoEdgeAi/. This also requires to install some dependencies for Python. The list of the required packages along with their versions is available as a text file in the package at location /FP-AI-MONITOR1_V1.0.0/Utilities/AI_resources/requirements.txt. The following command is used in the command terminal of the anaconda prompt or Ubuntu to install all the packages specified in the configuration file requirements.txt:
pip install -r requirements.txt
Info white.png Information
In this document all the steps presented are carried out with STM32CubeIDE but the same can also be done with the other two IDEs.

2. Condition monitoring

The following section shows how to set up a condition monitoring project. The following figure shows all the steps with brief details.

FP-AI-MONITOR1 neai lib generation flow.png

2.1. Data acquisition

FP-AI-MONITOR1 contains as a utility, a precompiled HSDatalog.bin binary file for FP-SNS-DATALOG1. The binary file is located under /FP-AI-MONITOR1_V1.0.0/Utilities/Datalog/ directory. This file can be used to program the STEVAL-STWINKT1B to enable the datalogging functionality. This section briefly describes how a user can use this binary to log the data using onboard sensors of STEVAL-STWINKT1B. Users can program the sensor-board using this file by simply performing a drag-drop action as shown in the figure below.

how to AD installing hsdatalog bin.png
Info white.png Information
For data logging using the high speed datalogger user needs a FAT32-FS formatted microSDTM card.

Once the sensor-board is programmed log the data using the following instructions.

  • Place a DeviceConfig.json file with the sensors configurations to be used for the datalogging in the root folder of the microSDTM card. Some sample .json files are provided in the package and are located in FP-AI-MONITOR1_V1.0.0/Utilities/Datalog/Sample_STWIN_Config_Files/. These files are to be precisely named DeviceConfig.json.
  • Insert the SD card into the STWIN board.
  • Reset the board. Orange LED blinks once per second. The custom sensor configurations provided in DeviceConfig.json are loaded from the file.
  • Press the [USR] button to start data acquisition on the SD card. The orange LED turns off and the green LED starts blinking to signal sensor data is being written into the SD card.
  • Press the [USR] button again to stop data acquisition. Do not unplug the SD card or turn the board off before stopping the acquisition otherwise the data on the SD card will be corrupted.
  • Remove the SD card and insert it into an appropriate SD card slot on the PC. The log files are stored in STWIN_### folders for every acquisition, where ### is a sequential number determined by the application to ensure log file names are unique. Each folder contains a file for each active sub-sensor called SensorName_subSensorName.dat containing raw sensor data coupled with timestamps, a DeviceConfig.json with specific information about the device configuration, necessary for correct data interpretation, and an AcquisitionInfo.json with information about the acquisition.
Info white.png Information
For details on how to use all the features of the provided HSDatalog.bin binary the users are invited to refer to the user manual of FP-SNS-DATALOG1.

2.1.1. Logging for normal and abnormal conditions

The anomaly detection library generation in NanoEdge AI Studio requires some example data for the normal and abnormal conditions to select an optimal algorithm and library for the anomaly detection for any given setup. Following the steps in this section the users can log the data for the normal and abnormal conditions. As the HSDatalog.bin enables us to log the data using multiple or all sensors on the board, we will use this feature to log the data for both ISM330DHCX and IIS3DWB sensors to later use which ever data we want. Take an example DeviceConfig.json file and make sure all the sensors and subsensors are disabled but specifying the values for isActive flag as false. Now enable the ISM330DHCX and IIS3DWB sensors by setting the value of isActive flag to true in accelerometer sub-senor configuration. Set the odr (output data-rate) for ISM330DHCX sensor to 1667 and Fs (full-scale) to 4 for both sensors. Insert the microSD card in sensor-board, and restart the board by pressing RESET button. You are ready to log the data now.

Warning white.png Warning
add the picture for the fan setup

Put the STWIN on the fan as shown in the figure above, while it is running in normal condition (with no problems). The fan being used in this demo has a handy controller which lets users to turn-on and turn-off the fan, as well as run it at one of the three speed settings. The following figure is showing the controller image.

how to AD fan controller.png

As the readers can see the settings are labeled on the controller are:

  • O : Fan is in Off state,
  • L : Fan is running at Low speed (1000 RPM),
  • M : Fan is running at Medium speed (1250 RPM), and
  • H : Fan is running at High speed (1500 RPM).

Start the fan at the low speed when the button is at L settings, when the fan is stable, start the data logging by pressing the user button on the STEVAL-STWINKT1B the green LED light will start to flash. Wait for 40 seconds, then press the USR Button again to stop the logging of the data, the green LED light will stop blinking. Repeat the process for medium and high speed. This will generate three datalogs for fan speeds L (1000RPM), M (1250RPM) and H (1500RPM), named as ST0001, ST0002, and ST0003 respectively. Now we can simulate the clogged condition by putting a piece of paper on the inlet side (where the air flow is inside) of the fan. Repeat the steps for the datalogging for logging data in abnormal conditions at the three speeds. This will generate three more data folders named as ST0004, ST0005, and ST0006 for speeds L (1000RPM), M (1250RPM) and H (1500RPM) respectively.

2.1.2. Data preparation for library generation with NanoEdge™ AI Studio

The data logged through the datalogger is in the binary format and is not user readable nor compliant with the NanoEdge™ AI Studio format as it is. To convert this data to a useful form, FP-AI-MONITOR1 provides Python™ utility scripts on the path /FP-AI-MONITOR1_V1.0.0/Utilities/AI-resources/NanoEdgeAi/.

The Jupyter Notebook NanoEdgeAI_Utilities.ipynb provides a complete example of data preparation for a three speed fan library generation running in normal and clogged condition. The script out of the package use the example data provided in the package. This section will guide users to use these scripts to generate an anomaly detection library using the six datalogs generated following the instruction in the section above.

The NanoEdge AI studio expects one normal and one abnormal file as input for anomaly detection library generation. Hence we need to combine the data in the three speeds for normal and clogged conditions. To use the provided scripts as it is without changing too much, we advise readers to rename and arrange their six data acquisitions as shown in the folder below.

how to AD datalogs grouping.png

Once the datalogs are renamed, arranged and grouped as shown in the figure above, copy and paste the folder named as Fan12CMNew at the path FP-AI-MONITOR1_V1.0.0/Utilities/AI_Resources/HSD_Logged_Data/Fan12CMNew/ and open the Jupyter Notebook named NanoEdgeAI_Utilities.ipynb. You need to edit the code as shown in the screenshot below:

how to AD adding path to new dataset.png

After making these changes, save the file and run all the cells by going to the Cell menu in the toolbar and select Run All the sections option to run all the sections in the notebook, as shown in the figure below.

how to AD running all sections from jupyter.png

If there are no errors, after the last section is run, as a result there will be two files generated in the folder Fan12CMNew named as clogged_WL1024_segments.csv and normal_WL1024_segments.csv.

This script enables users to generate the input files for NanoEdge AI studio very easily by just changing the variable sensorName. The default variables for the frame length and stride are set for both sensors to result in non-overlapping windows of length 1024 and 4096 for ISM330DHCX and IIS3DWB sensor.

Note : In addition to this Jupyter Notebook there is a HSD_2_NEAISegments.py is provided if user wants to prepare segments for a given data acquisition. This script is used by issuing the following command for a data acquisition with ISM330DHCX_ACC sensor. 

python HSD_2_NEAISegments.py ../Datasets/Fan12CM/ISM330DHCX/normal/1000RPM/

This command will generate a file named as ISM330DHCX_Cartesiam_segments_0.csv using the default parameter set. The file is generated with segments of length 1024 and stride 1024 plus the first 512 samples are skipped from the file.

2.2. Generating a condition monitoring library

The process of generating the libraries with NanoEdge™ AI Studio consists of six steps. These steps can be seen in the figure below.

FP-AI-MONITOR1-neai lib generation.png
  1. Hardware description
    • Choosing the target platform or a microcontroller type: STEVAL-STWINKT1B Cortex-M4
    • Maximum amount of RAM to be allocated for the library: Usually a few Kbytes is enough (but it depends on the data frame length used in the process of data preparation, 32 Kbytes is a good starting point) so we start from 32 Kbytes.
    • Sensor type : select 3-axis accelerometer for this project
  2. Providing the sample contextual data for normal segments generated in the previous section. This data will be used to find the best parameters to have the best performances for the given setup. i.e. the normal_WL1024_segments.csv.
  3. Providing the sample contextual data for abnormal segments generated in the previous section for fine tuning the model to have best performance on the given set up. i.e. clogged_WL1024_segments.csv.
  4. Benchmarking of available models and choose the one that complies with the requirements.
  5. Validating the model for learning and testing through the provided emulator which emulates the behavior of the library on the edge.
  6. The final step is to compile and download the libraries. In this process, the flag "-mfloat-abi" has to be checked for using libraries with hardware FPU. All the other flags can be left to the default state.

Note : For using the library with μKeil also check -fshort-wchar in addition to -mfloat-abi.


The detailed documentation on the NanoEdge™ AI Studio regarding the AI library generation and working with these libraries can be found here.

2.3. Installing the NanoEdge™ Machine Learning library

Once the libraries are generated and downloaded from NanoEdge™ AI Studio, the next step is to link these libraries to FP-AI-MONITOR1 and run them on the STWIN. The FP-AI-MONITOR1, comes with the library stubs in the place of the actual libraries generated by NanoEdge™ AI Studio. This is done to simplify the linking process of the generated libraries. In order to link the generated libraries, the user needs to copy the generated libraries and replace the existing stub/dummy libraries libneai.a and header files NanoEdgeAI.h present in the folders lib and Inc, respectively as shown in the figure below. Both of these folders can be found at the path /FP_AI_MONITOR1_V1.0.0/Middlewares/ST/NanoEdge_AI_Library/.

FP-AI-MONITOR1-linking NEAI library.png

Once these files are copied, the project must be reconstructed and programmed on the sensor board to link the libraries correctly. For this, the user must open the project in STM32CubeIDE located in the /FP-AI-MONITOR1_V1.0.0/Projects/STWINL4R9ZI-STWIN/Applications/FP-AI-MONITOR1/STM32CubeIDE/ folder and double click .project file as shown in the figure below.

FP-AI-MONITOR1-opening IDE project.png

Then build and install the project in STEVAL-STWINKT1B sensor-board by pressing the play button as shown in the figure below.

FP-AI-MONITOR1-building and installing the project.png

A message saying Download verified successfully indicates the new firmware is programmed in the sensor-board.

Info white.png Information
NOTE : To be absolutely sure that a new software is installed in STEVAL-STWINKT1B, user can issue command $ info. This will show the compile time and user can confirm if the sensor-board is programmed with new or old binary. If executing the info command shows an older time and date, try source clean option from the CubeIDE and then rebuild and flash the the project by pressing the play button.

2.4. Testing the NanoEdge™ AI Machine Learning library

Once the STWIN is programmed with the FW containing a valid library, the condition monitoring libraries are ready to be tested on the sensor board. The libraries can now to used to perform the condition monitoring using the STEVAL-STWINKT1B programmed with FP-AI-MONITOR1.

2.4.1. Connecting the command line interface (CLI) application

FP-AI-MONITOR1 comes with an interactive CLI application. The CLI uses Virtual COM port over USB to interact with the sensor-board. This section shows how to connect the sensor-board using this and enable the CLI.

On a windows powered machine start the Tera Term, select the proper connection (featuring the STMicroelectronics name) as shown in the figure below,

tera term connection port choice.png

Set the parameters:

  • Terminal
    • [New line]:
      • [Receive]: CR
      • [Transmit]: CR
tera term connection new line.png
  • Serial
    The interactive console can be used with the default values.

Restart the board by pressing the RESET button. The following welcome screen is displayed on the terminal.

FP-AI-MONITOR1 Console Welcome Message

From this point, the users can start entering the commands directly or type help to get the list of available commands along with their usage guidelines.

2.4.2. Learning normal conditions

To perform the on device learning the user can start the learning phase by simply entering the command start neai_learn but before doing this the user needs to configure the sensor to use. To achieve the best performances the user should use the same sensor configurations as used for the data for data acquisition, i.e. enable ISM330DHCX sensor with ODR = 1666 and FS = 4, and make sure the other sensors are disabled. This can be achieved by issuing the command sequence shown in the following figure.

File:how to AD configuring sensor.png

Once the sensor is configured the user needs to learn the normal running conditions for the fan at all the three speeds by issuing the command start neai_learn. The learning can be stopped by simply pressing the ESC button on the keyboard or by pressing the USR button on the sensor-board. Every library generated from NanoEdge AI Studio comes with some guidelines on how many minimum training samples are to be used to get optimal performance. It is recommended that user uses atleast three to five time more samples than reported. For example if the minimum iteration value is 40, the user has to perform learning at each of the three speeds using 120 to 200 samples.

Info white.png Information
Bonus: FP-AI-MONITOR1 has a command call neai_set signals, which enables users to run the learn and detect phases for provided number of iterations. So, if the user issues command neai_set signals 120, before issue start neai_learn command, the learning will automatically stop after 120 iterations.

2.4.3. Performing condition monitoring

Once the normal state has been learned the condition monitoring or detect phase can be started by issue start neai_detect command. During this process, the similarity of the vibration pattern is compared with the learned patterns, and whenever the similarity is below than the set threshold (default value for threshold is 90%), a message is printed in the console showing the occurrence of the anomaly. This threshold can be changed and set using neai_set threshold <threshold_value> command, where threshold_value can be an integer value between [0 and 100].

Info white.png Information
Note : The NanoEdge™ AI library comes with an incremental learning capability, so the learning and detection phases can be toggled depending on whether the needs to be without forgetting the previous learning or not.

2.5. Additional parameters in condition monitoring

For user convenience, the CLI application also provides handy options to easily fine-tune the inference and learning processes. The list of all the configurable variables is available by issuing the following command: 

$ neai_get all
NanoEdgeAI: signals = 0
NanoEdgeAI: sensitivity = 1.000000
NanoEdgeAI: threshold = 95
NanoEdgeAI: timer = 0

Each of the these parameters is configurable using the neai_set <param> <val> command.

This section provides information on how to use these parameters to control the learning and detection phase. By setting the "signals" and "timer" parameters, the user can control how many signals or for how long the learning and detection is performed (if both parameters are set the learning or detection phase stops whenever the first condition is met). For example, to learn 10 signals, the user issues this command, before starting the learning phase as shown below. 

$ neai_set signals 10
NanoEdge AI: signals set to 10
$ start neai_learn
NanoEdgeAI: starting
$ {"signal": 1, "status": success}
{"signal": 2, "status": success}
{"signal": 3, "status": success}
:
:
{"signal": 10, "status": success}
NanoEdge AI: stopped

If both of these parameters are set to "0" (default value), the learning and detection phases will run indefinitely.

The threshold parameter is used to report any anomalies. For any signal which has similarities below the threshold value, an anomaly is reported. The default threshold value used in the CLI application is 90. Users can change this value by using neai_set threshold <val> command.

Finally, the sensitivity parameter is used as an emphasis parameter. The default value is set to 1. Increasing this sensitivity will mean that the signal matching is to be performed more strictly, reducing it will relax the similarity calculation process, i.e. resulting in higher matching values.

Info white.png Information
Note: For the best performance, the user must expose all the normal conditions to the sensor-board during the learning, and library generation process, i.e. in case of motor monitoring, the required speeds and ramps that need to be monitored must be exposed.

For further details on how NanoEdge™ AI libraries work users are invited to read the detailed documentation of NanoEdge™ AI Studio.

3. Documents and related resources

Following are some resources which can be helpful for the readers.

Retrieved from "https://wiki.st.com/stm32mcu/index.php?title=AI:How_to_perform_anomaly_detection_using_FP-AI-MONITOR1&oldid=15119"