|
Delivery for this distribution is being prepared |
In this guide, you will learn how to get started using the Function Pack for Predictive Maintenance FP-AI-NanoEdg1 and use it as your first project to detect anomalies in your setup using accelerometer data.
This Function Pack supports two STM32 boards, STWIN and STM32L562E. Following the steps provided in this page, a user will be able to create the data logs for their setups, and generate EdgeAi libraries using Cartesiam NanoEdge AI Studio. Once these libraries are generated, the user can use these libraries by plugging them into the provided FP and can learn the nominal behavior of his setup by issuing commands through a command line interface (CLI). After training phase the monitoring can be started and any anomalies from the normal behavior are reported on the CLI as well as on the LCD (in case of L5 board).
1. What you will learn
- How to log the motion sensor data.
- How to parse and read the motion sensor data.
- How to generate the suitable libraries for your sample data set using Cartesiam NanoEdge AI Studio.
- Learn and detect the anomalies and controlling the hyper parameters of learning and detection.
2. Condition Monitoring using NanoEdge AI Machine Learning Library
This section provides complete method to set up an application for condition monitoring using FP-AI-NANOEDG1. The steps are provided with brief details in the diagram below.
The details on how to set up the sensor board with function pack as well as data collection have been already provided in the sections above. Following sections provide the details on library generation, installation, training and testing.
2.1. Generating a Condition Monitoring Library
The AI libraries for the condition monitoring for this function pack are generated and provided by NanoEdge AI Studio, a solution provided by Cartesiam. This section provides step by step guide on how to generate these condition monitoring libraries, installing them on the sensor node by linking them to the provided project in FP-AI-NANOEDG1 and then using them to perform the condition monitoring on the edge, by running first in training and then detection mode.
2.1.1. Data Logging for Normal and Abnormal Conditions
The library generation requires some sample data to be provided to the NanoEdge AI Studio. This data will provide the context of the set up to be monitored. The first step is to log the data for the normal and abnormal conditions using the data logging application provided in this function pack. Note that this data is to generate the libraries, so the users are advised to record data by exposing all the possible normal and any known abnormal conditions.
2.1.2. Data Preparation for Library Generation with NanoEdge AI Studio
The data logged through the datalogger is in the binary format and is not user readable in its current condition. In order to convert this data to useful form, FP-AI-NANOEDG1 provides python utility scripts. These scripts along with example usage can be found in a Parser_for_HS_Logged_Data.ipynb , a Python Jupyter Notebok, located on the path FP-AI-NANOEDG1/Utilities/AI-resources/DataLog/ as shown in the figure below.
2.1.3. Library Generation using NanoEdge AI Studio
The process of generating the libraries with Cartesiam NanoEdge AI studio consists of five steps.
- Hardware description
- Choosing a microcontroller type : ARM® Cortex® M33
- Maximum amount of RAM to be allocated for the library: Usually a few KBs will suffice (but it depends on the dataframe length used in the process of data preparation).
- Sensor type : 3-axes accelerometer
- Providing the sample contextual data to adjust and gauge the performance of the chosen model. This step will require data for:
- Nominal or Normal case
- Abnormal case
- Benchmarking of available models and choosing the one which complies with the requirements.
- Validating the model for learning and testing through the provided emulator which emulates the behavior of the library on the edge.
- The final step is to compile and download the libraries. In this process the flag mfloatabi has to be unchecked for using libraries with hardware FPU.
The detailed documentation on the NanoEdge AI Studio is available here NanoEdge AI Studio Documentation.
2.2. Installing the NanoEdgeAI Machine Learning Library
Once the libraries are generated and downloaded from Cartesiam NanoEdge AI Studio, the next step is to incorporate these libraries to FP-AI-NANOEDG1. The FP-AI-NANOEDG1, comes with the library stubs in the place of the actual libraries generated by NanoEdge AI studio. This is done in order to make it easy for users to link the generated libraries and have a place holder for the libraries which are generated through following the steps provided in the previous section. In order to link the actual libraries, the user needs to copy the generated libraries and replace the existing stub/dummy libraries and header files NanoEdgeAI.h, and libneai.a files present in the folders lib, and bin, respectively. The relative paths of these folders are /FP_AI_NANOEDG1/Middlewares/Third_Party/Cartesiam_NanoEdge_AI_Library/ as shown in the figure below.
Once these files copied, the project must be reconstructed and flashed on the sensor board in order to link the libraries. For this user needs to open the .project file from the FP-AI-NANOEDG1 folder, located at the path FP_AI_NANOEDG1/Projects/STM32L562E-DK/Applications/NanoEdgeConcole/STM32CubeIDE/ as shown in the step 2 in figure below.
To install the new firmware after linking the library, connect the sensor board and rebuild the project using the play button. See the console for the outputs and wait for the Build and Download success message as shown in step 4 in the figure below.
2.3. Testing the NanoEdgeAI Machine Learning Library
Once the Downoad verified successfully message appears, the generated AI libraries are linked to the firmware on the sensor board and are ready to test.
Note that in its current form the libraries have zero knowledge of the normal and abnormal conditions, and to perform the condition monitoring effectively, it is important to learn the normal behavior of the setup/machine to be modeled. This can be done by starting the learning process. The learning phase can be started by issuing start neai_learn command in console. Starting this command will show the process of learning and will show a message on the console every time learning is performed on a new signal, as shown in below code snippet.
$ start neai_learn NanoEdgeAI: starting $ 1 signal learning: SUCCESS. 2 signal learning: SUCCESS. 3 signal learning: SUCCESS. 4 signal learning: SUCCESS. 5 signal learning: SUCCESS. 6 signal learning: SUCCESS.
The process can be stopped either by issuing the
$ stop
command or by simply pressing the ESC key on the keyboard.
The alternate way to start the learning process is to long press the user button, as showed in the splash screen of the sensor node. This button can be used to start the learning in standalone as well as in computer operated mode. The process can be stopped by simply pressing the user button again.
Once the normal conditions are learned, the user can start the condition monitoring process by starting the neai_detect mode. This mode can be started by issuing the command
$ start neai_detect NanoEdgeAI: starting
or by short pressing the user button on the sensor board. The LCD will show the detection mode is activated along with the status through the LED in the screen, Green for Normal, Orange for Anomaly, and Red for Failure, as shown in the screen shots in figure above.
2.3.1. Some Hyper-parameters in Condition Monitoring
For the convenience of the users CLI also provides some handy options to run the inference and learning process of the condition monitoring easily. Users can see all the variables they can play with by issuing following command:
$ neai_get all NanoEdgeAI: signals = 0 NanoEdgeAI: sensitivity = 1.000000 NanoEdgeAI: threshold = 95 NanoEdgeAI: timer = 0
Each of the these parameters can be configured using neai_set <param> <val> command. The details are provided in the command table above.
This section provides small details on neai_set signals <nr of signals> . Using this command a user can start the learning or condition monitoring mode for a given number of samples. For example, if the user wants to learn on 100 signals, they can issue this command, before issuing the learn command. In the following snippet an example is presented where the learning is performed for 10 signals.
$ neai_set signals 10 NanoEdgeAI: signals set to 10 $ start neai_learn NanoEdgeAI: starting $ 1 signal learning: SUCCESS. 2 signal learning: SUCCESS. 3 signal learning: SUCCESS. 4 signal learning: SUCCESS. 5 signal learning: SUCCESS. 6 signal learning: SUCCESS. 7 signal learning: SUCCESS. 8 signal learning: SUCCESS. 9 signal learning: SUCCESS. 10 signal learning: SUCCESS. NanoEdge AI: stopped
The threshold parameter is used to report the anomalies. For any signal which has similarities below the threshold value is reported as anomaly. The default value used in the CLI application is 95. Users can change this value by using neai_set threshold <val> command.
The sensitivity parameter is used as an emphasis parameter. The default value is set to 1. Increasing this sensitivity will mean that the matching of the signals is to be performed more strictly, reducing it will also reduce the matching calculation process, i.e. resulting is higher matching values.
For details users are invited to read the detailed documentation of NanoEdge AI studio here.
3. Requirements
3.1. Hardware
- STWIN or STM32L562E
The STWIN (STEVAL-STWINKT1) and STM32L562E (L5) are two STM32 boards with environment sensors whose a three axis accelerometer.
- STLINK-V3MINI (only for the STWIN)
The STLINK-V3MINI is a stand-alone debugging and programming mini probe for STM32 microcontrollers.
- Micro SD card between 4GB and 16GB
The micro SD card will be using for the data collection.
- Laptop/PC with Windows 7 or later
- USB cables with micro USB connector
3.2. Software
- STM32 CubeIDE
Download STM32 CubeIDE from ST website. This software is an IDE for STM32, its allows the development and the compilation of firmwares.
- Optional: STM32 ST-LINK Utility (STSW-LINK004) or STM32CubeProgrammer
STM32 ST-LINK Utility and STM32CubeProgrammer allow the programming of STM32 targets and can be useful for some cases (example : download bineries on STM32 targets or erase their flash).
- Tera Term
You can easily download Tera Term form the internet.
- Cartesiam NanoEdgeAIStudio
You can get a pro licence with your company. For that contact support@cartesiam.com. Otherwise, you can download NanoEdge AI Studio for free by filling out the following form: https://share.hsforms.com/1vVgNZ5JfQAWeJs10Yw6oHQ2a3c9.
At this moment of the tutorial, if you want more information about this software see the NanoEdge AI Studio Documentation.
- Jupyter notebook
Allows to execute python code. You can get Jupyter notebook by downloading Anaconda.
- Download or clone the FP-AI-NanoEdg1 Function Pack
Download the latest version of FP-AI-NanoEdg1 function pack from ST website or clone the project into your STM32CubeIDE workspace directory.
The command to clone FP-AI-NanoEdg1 is the following:
$ cd <userName>/STM32CubeIDE/workspace_<version> $ git clone ssh://gitolite@codex.cro.st.com/ai-poc/STM32_PREDMNT1_TMP.git
4. Step 1 : Programming the STWIN with FP-AI-NanoEdg1
In order to flash the STWIN with the FP-AI-NanoEdg1 you need to use the STLINK-V3MINI. The connection steps are the following:
- Connect one side of the programming cable to STLINK-V3MINI and the other side to STWIN
- Connect STWIN to a micro USB linked to your PC
- Connect STLINK-V3MINI to a micro USB linked to your PC
Once the cabling is done you have several options in order to flash the FP-AI-NanoEdg1 firmware inside the STWIN.
4.1. Option 1: Drag and drop the binary (if the binary is already generated)
Drag and drop the binary (.bin) located in:
FP_AI_NANOEDG1\Projects\<boardName>\Applications\NanoEdgeConsole\Binary
4.2. Option 2: Flash the binary with STM32 ST-LINK Utility (if the binary is already generated)
- Open STM32 ST-LINK Utility
- Click on Connect to the target
- Click on Program verify
- Browse the binary (.bin) located in:
FP_AI_NANOEDG1\Projects\<boardName>\Applications\NanoEdgeConsole\Binary
- Load the firmware with the Start button
4.3. Option 3: Flash the binary with STM32 CubeProgrammer (if the binary is already generated)
- Open STM32 CubeProgrammer
- Click on Connect (green button)
- Go in the Erasing & Programming (see in the menu)
- In File path (Download part) browse the binary (.bin) located in:
FP_AI_NANOEDG1\Projects\<boardName>\Applications\NanoEdgeConsole\Binary
- Load the firmware with the Start Programming blue button
4.4. Option 4: Compile the FP-AI-NanoEdg1 project with STM32CubeIDE
- Open STM32 CubeIDE
- Import a STM32CubeIDE project
- Click on File --> Import
- Select General --> Existing Projects into Workspace
- Browse the STM32CubeIDE project according to the board you use :
STWIN : STM32_PREDMNT1_TMP\FP_AI_NANOEDG1\Projects\STM32L4R9ZI-STWIN
STM32L562E : STM32_PREDMNT1_TMP\FP_AI_NANOEDG1\Projects\STM32L562E-DK
- Click on Finish
If you have imported the two STM32CubeIDE projects you will get the following organization:
- Select and build (hammer icon) the STM32CubeIDE project you want to flash
- Program the project on STWIN with the run button (play green button)
5. Step 2 : How to make the data logging
Once the STWIN is programmed, you can remove the STLINK-MINI and let only the STWIN connected to the PC. The data will be logging inside a micro SD card.
5.1. Connection between the sensor and the Tera Term application
Once the micro SD card is installed in the SD card slot of your board, you can connect the STWIN through serial port to the Tera Term Application. For the configuration of the application follow these steps:
- Open Tera Term
- Open a new connection: File --> New connection
- Select Serial and chose the port number relative to the STWIN (STMicroelectronics Virtual COM Port)
- Click on OK
You will see the following terminal with a $ which indicates you can write a command:
5.2. Use CLI commands to configure sensors for data logging
In this part, we will explained the CLI commands which allows to configure sensors.
- Help
Allows to see all the possible commands with their parameters.
$ help
- Sensor info
Allows to get a list of all supported sensors (and sub-sensors) by your boad and their ID.
$ sensor_info
For example, with the STWIN, you will get the following information about supported sensors and sub sensors:
Here, the command shows us that the STWIN has two sensors: IIS3DWB (ID = 0) and ISM330DHCX (ID = 1). For the predictive maintenance application, we collect data from an accelerometer. In that case, we can only use the accelerometer (ID = 0) from the ISM330DHCX sensor (ID = 1).
- Sensor get
Allows to get the value of a parameter from a sensor with its sensor ID.
$ sensor_get <sensor_id.subsensor_Id> all
For example, with the STWIN, we can get the value of all the parameters of the accelerometer from ISM330DHCX sensor (ID = 1.0) with the following command:
$ sensor_get 1.0 all
Here we can see that the sub sensor is activated, has an ODR at 1667 Hz and a full scale at 16 mg.
- Sensor set
Allows to set the required value of a parameter for the datalogging with sensor id.
$ sensor_set <sensor_id.subsensor_Id> <param> <val>
Example: With the STWIN, we can change the full scale value of the accelerometer from ISM330DHCX sensor (ID = 1.0) with the following command:
$ sensor_set 1.0 fullScale 4.0
5.3. Use CLI commands to launch data logging
In this part, we will explained the CLI commands which allows to make data logging.
- Start datalog
Allows to start to logging of the data. You can launch this command once the configuration of your sensor is finished and your system is ready in the desired behaviour (normal or abnormal condition).
$ start datalog
- Stop
Allows to stop the data logging process once the data is logged for the required time (1min for example). You can also use the Esc button.
$ stop
6. Step 3 : How to parse and read the motion sensor data
6.1. Take the data from the SD card
Once you have finish with the data collection, remove the micro SD card from the STWIN and see its capacity on your PC. If you made four recording, you will get something like the following example:
Each data folder has the following organization:
STWIN_00X/ ├── DeviceConfig.json └── ISM330DHCX.dat
- Json file
The json file corresponds to all configurations about the STWIN (sensor and sub sensor activated, ODR or full-scale values, …).
- Dat file
The dat file corresponds to the X, Y, Z values of the accelerometer. However, we can’t read directly the data so we need to parse the data in order to understand the meaning in a human way.
6.2. Parse the data
To parse the motion sensor data you need to use the python script called Parser_for_HS_Logged_Data.ipynb that you can execute with Jupyter notebook. The python script is available inside the function pack at the following path:
STM32_PREDMNT1_TMP\FP_AI_NANOEDG1\Utilities\AI_ressources\DataLog.
The python script has two parts:
- The data parser
- Prepare CSVs for NanoEdgeAI
The second part will be used after, for now we focus on only on the first part which contains the two following blocs:
To parse your data follow these steps:
- Put your data folder inside the Sample- Datalogs folder
The organization will be the following:
Data_log/ ├── Sample-DataLogs │ ├──STWIN_00X │ ├── DeviceConfig.json │ ├── ISM330DHCX.dat │ ├──STWIN_00X │ ├── DeviceConfig.json │ ├── ISM330DHCX.dat │ ├── ...
- Chose one of the two blocs on Jupyter (part 1.1 or part 1.2)
- If it's necessary change the python script according to your need (directory number, directory path,...)
- Execute the code
For each data folder, the code will generate an ISM330DHCX_ACC.csv file
Data_log/ ├── Sample-DataLogs │ ├──STWIN_00X │ ├── DeviceConfig.json │ ├── ISM330DHCX.dat │ ├── ISM330DHCX_ACC.csv │ ├──STWIN_00X │ ├── DeviceConfig.json │ ├── ISM330DHCX.dat │ ├── ISM330DHCX_ACC.csv │ ├── ...
7. Step 4: How to generate normal and abnormal databases for NanoEdgeAI
Now, we can gather all your normal data inside a same csv file and make the same thing for your abnormal data. To do so we will use the second part of the python script Parser_for_HS_Logged_Data.ipynb (Prepare CSVs for NanoEdgeAI) which contains the two following blocs:
To generate your two databases, follow these steps:
- Put your csv data files inside the good folders (normal or abnormal)
The organization will be the following (for example) :
Data_log/ ├── Sample-DataLogs │ ├── Normal folder │ ├──STWIN_001 │ ├── DeviceConfig.json │ ├── ISM330DHCX.dat │ ├── ISM330DHCX_ACC.csv │ ├──STWIN_002 │ ├── Abnormal folder │ ├──STWIN_003 │ ├──STWIN_004
- Choose one of the two blocs on Jupyter (part 2.1 or part 2.2)
- If it's necessary change the python script according to your need (directory number, directory path,...)
- Execute the code
The code will generate two csv files called abnormalSegments.csv and normalSegments.csv inside the Data log folder.
Data_log ├── abnormalSegments.csv ├── normalSegments.csv
8. Step 5: How to generate the libraries with NanoEdge AI Studio
For this step you can refer to the part IV of the Cartesiam’s documentation or the Cartesiam’s tutorial YouTube.
Nevertheless, we will give you some precisions about some steps :
- Create new project
In target select the microcontroller type according to the board your using. That is to say :
- ARM Cortex M4 for STWIN
- ARM Cortex M33 for L5
- Importing signal files
Select the csv files generated by Parser_for_HS_Logged_Data.ipynb (see step 4):
- normalSegments.csv as regular signals
- abnormalSegments.csv as abnormal signals
- Compiling and downloading the NanoEdge AI Library
In compilation flags don’t select -mfloat-abi
At the end the user will receive a zip folder called NanoEdgeAI_Library.zip