|
a draft is started based on the Getting Started with FP-AI-NANOEDG1 |
Condition monitoring is a major component of the predictive maintenance systems, allowing production performance improvement, maintenance cost reduction, and a drastic decrease of the downtime due to routine maintenance. The FP-AI-NANOEDG1 function pack helps to jump-start the implementation and development of condition monitoring applications designed with the NanoEdge™ AI Studio solution from Cartesiam (a member of the ST Partner program).
NanoEdge™ AI Studio simplifies the creation of autonomous Machine Learning libraries with the possibility of running not just inference but also training on the edge. It facilitates the integration of predictive maintenance capabilities as well as the security and detection with sensor patterns self-learning and self-understanding, exempting users from special skills in mathematics, Machine Learning, data science, or creation and training of Neural Network.
FP-AI-NANOEDG1 covers the entire design of the Machine Learning cycle from the data set acquisition to the integration of NanoEdge™ AI Studio generated libraries on a physical node. It runs the inference in real-time on an STM32L4R9ZI ultra-low-power microcontroller (Arm® Cortex®-M4 at 120 MHz with 2 Mbytes of Flash memory and 640 Kbytes of SRAM), taking physical sensor data as input. The SensorTile wireless industrial node (STEVAL-STWINKT1B) embeds industrial-grade sensors, including 6-axis IMU, 3-axis accelerometer, and vibrometer to catch any movement precisely, a very high-frequency digital and analog microphone as well as ultrasound spectra enabling fine variation monitoring and also high-precision temperature, pressure, and humidity sensor for challenging industrial environment. The NanoEdge™ library generation itself is out of the scope of this function pack and must be generated using NanoEdge™ AI Studio.
FP-AI-NANOEDG1 implements a wired interactive CLI to configure the node, record the data, and manage the learn and detect phases. For simple operation in the field, a standalone battery-operated mode allows basic controls through the user button, without using the console.
This article provides an overview of the following topics:
- Overview of the required hardware and software,
- Setting up the hardware and software components,
- Performing the vibration sensor data collection using the provided function pack,
- Generating, Installing, and testing the NanoEdge™ AI machine learning libraries on the sensor node STEVAL-STWINKT1B using the provided function pack, and
- Some links to useful online resources, which can help a user to better understand the project and then customize the project for his own needs.
1. Hardware and software overview
1.1. SensorTile Wireless Industrial Node Evaluation Kit STEVAL-STWINKT1B
The STWIN SensorTile wireless industrial node (STEVAL-STWINKT1B) is a development kit and reference design that simplifies the prototyping and testing of advanced industrial IoT applications such as condition monitoring and predictive maintenance. It is powered with Ultra-low-power Arm® Cortex®-M4 MCU at 120 MHz with FPU, 2048 kbytes Flash memory (STM32L4R9). Other than this, STWIN SensorTile is equipped with a microSD™ card slot for standalone data logging applications, a wireless BLE4.2 (on-board) and Wi-Fi (with STEVAL-STWINWFV1 expansion board), and wired RS485 and USB OTG connectivity, as well as Brand protection secure solution with STSAFE-A110. In terms of sensor, STWIN SensorTile is equipped with a wide range of industrial IoT sensors including:
- an ultra-wide bandwidth (up to 6 kHz), low-noise, 3-axis digital vibration sensor (IIS3DWB)
- a 6-axis digital accelerometer and gyroscope iNEMO inertial measurement unit (IMU) with machine learning (ISM330DHCX)
- an ultra-low-power high-performance MEMS motion sensor (IIS2DH)
- an ultra-low-power 3-axis magnetometer (IIS2MDC)
- a digital absolute pressure sensor (LPS22HH)
- a relative humidity and temperature sensor (HTS221)
- a low-voltage digital local temperature sensor (STTS751)
- an industrial-grade digital MEMS microphone (IMP34DT05), and
- a wideband analog MEMS microphone (MP23ABS1)
Other attractive features include:
- – a Li-Po battery 480 mAh to enable standalone working mode
- – STLINK-V3MINI debugger with programming cable to flash the board
- – a Plastic box for ease of placing and planting the SensorTile on the machines for condition monitoring. For further details, the users are advised to visit this link
1.2. FP-AI-NANOEDG1 V2.0 Software Description
The STM32Cube function packs leverage the modularity and interoperability of STM32 Nucleo and expansion boards running STM32Cube MCU Packages and Expansion Packages to create functional examples representing some of the most common use cases in certain applications. The function packs are designed to fully exploit the underlying STM32 ODE hardware and software components to best satisfy the final user application requirements.
Function packs may include additional libraries and frameworks, not present in the original STM32Cube Expansion Packages, which enable new functions and create more targeted and usable systems for developers.
STM32Cube ecosystem includes:
- A set of user-friendly software development tools to cover project development from the conception to the realization, among which are:
- STM32CubeMX, a graphical software configuration tool that allows the automatic generation of C initialization code using graphical wizards
- STM32CubeIDE, an all-in-one development tool with peripheral configuration, code generation, code compilation, and debug features
- STM32CubeProgrammer (STM32CubeProg), a programming tool available in graphical and command-line versions
- STM32CubeMonitor (STM32CubeMonitor, STM32CubeMonPwr, STM32CubeMonRF, STM32CubeMonUCPD) powerful monitoring tools to fine-tune the behavior and performance of STM32 applications in real-time
- STM32Cube MCU & MPU Packages, comprehensive embedded-software platforms specific to each microcontroller and microprocessor series (such as STM32CubeL4 for the STM32L4+ Series), which include:
- STM32Cube hardware abstraction layer (HAL), ensuring maximized portability across the STM32 portfolio
- STM32Cube low-layer APIs, ensuring the best performance and footprints with a high degree of user control over the HW
- A consistent set of middleware components such as FreeRTOS™, USB Device, USB PD, FAT file system, Touch library, Trusted Firmware (TF-M), mbedTLS, Parson, and mbed-crypto
- All embedded software utilities with full sets of peripheral and applicative examples
- STM32Cube Expansion Packages, which contain embedded software components that complement the functionalities of the STM32Cube MCU & MPU Packages with:
- Middleware extensions and applicative layers
- Examples running on some specific STMicroelectronics development boards
To access and use the sensor expansion board, the application software uses:
- STM32Cube hardware abstraction layer (HAL): provides a simple, generic, and multi-instance set of generic and extension APIs (application programming interfaces) to interact with the upper layer applications, libraries, and stacks. It is directly based on a generic architecture and allows the layers that are built on it, such as the middleware layer, to implement their functions without requiring the specific hardware configuration for a given microcontroller unit (MCU). This structure improves library code reusability and guarantees easy portability across other devices.
- Board support package (BSP) layer: supports the peripherals on the STM32 Nucleo boards.
The top-level architecture of the FP-AI-NANOEDG1 V2.0 function pack is shown in the following figure.
2. Project setup with example application
2.1. HW prerequisites and setup
To use the FP-AI-NANOEDG1 V2.0 function pack on STEVAL-STWINKT1B, following hardware items are required:
- STEVAL-STWINKT1B development kit board,
- Windows® powered laptop/PC (Windows® 7, 8, or 10),
- A USB cable to connect the PC to the Micro-B USB connector (CN17) on the board,
- A microSD™ card for data logging,
2.2. Software requirements
2.2.1. FP-AI-NANOEDG1 V2.0
- Download the FP-AI-NANOEDG1 V2.0, package from ST website, extract and copy the .zip file contents into a folder on your PC. The package contains binaries and source code with project files for three IDEs (STM32CubeIDE, Keil, and IAR), for sensor board STEVAL-STWINKT1B.
2.2.2. An IDE
- One of the following IDEs must be installed:
- STMicroelectronics - STM32CubeIDE version 1.4.2
- IAR Embedded Workbench for Arm (EWARM) toolchain V8.50.5 or later
- RealView Microcontroller Development Kit (MDK-ARM) toolchain V5.31
2.2.3. STM32 ST-LINK Utility
- A software interface for programming STM32 microcontrollers. This software is only required to install the prepared binaries provided in the function pack folder at start. Note that this software will be used by not just users to install the already provided binaries but by STM32CubeIDE also to program the boards with the generated binaries.
- This software can be downloaded from STSW-LINK004.
2.2.4. Tera Term
- Tera Term is an open source and freely available software terminal emulator, which is used to host the CLI of the FP-AI-NANOEDG1 through serial connection.
- Users can download and install the latest version available from Tera Term.
2.2.5. NanoEdge AI Studio
The function pack is to be powered by Cartesiam. For this, the user needs to generate the libraries from the NanoEdge AI Studio and then embed these libraries in the FP-AI-NANOEDG1. The Studio can be downloaded from this link. Cartesiam provides a free evaluation version for the STMicroelectronics boards.
2.3. FP-AI-NANOEDG1 console application
FP-AI-NANOEDG1 V2.0 provides an interactive command-line interface (CLI) application. This CLI application equips a user with the ability to configure and control the sensor node, to log the vibration data using industrial-grade sensors onboard, which will be used to generate the Cartesiam NanoEdgeTM AI libraries, as well as to perform learning and (anomaly) detection operations of these AI libraries on the edge. The following sections provide a small guide on how a user can install this CLI application on a sensor board and control it through the CLI from a PC.
2.3.1. Get the function pack
The first step is to get the function pack. The FP-AI-NANOEDG1 V2.0 can be downloaded from ST website. Once the pack is downloaded, the user can unpack/unzip it and copy the contents to a folder on the PC. The steps of the process along with the contents of the folder are shown in the following image.
2.3.2. Installing the CLI application on the sensor board STEVAL-STWINKT1B
Once the package has been downloaded and unpacked, the next step is to program the sensor node with the binary of the function pack. For the convenience of the users, the function pack is equipped with a pre-built binary file of the project. The sensor board can be very easily programmed with the provided binary by simply performing a drag-and-drop action as shown in the figure below.
2.3.3. Use the command line interface (CLI)
Once the sensor board is programmed with the binary of the project, the next step is to set up the serial connection of the board with the PC through Tera Term. The first step is to create a new connection. This can be done either by navigating from the toll bar of Tera Term or by entering the key combination Alt + N . This will show a dialog as shown in the figure below. Select the port on which your sensor board is attached. In the following example, communication port 28 is selected with title COM28: STMicroelectronics - STLink Virtual COM Port.
The next step is to configure the connection. Following image shows the configurations to be chosen for the serial connection to have an effective communication between computer and sensor board.
Once the connection is made the user will see a message
Console command server. Type 'help' to view a list of registered commands. $
Typing help will show the list of all available commands as presented in the table below.
| Command name | Command string | Note |
|---|---|---|
| CS1 - Generic Commands | ||
| help | help | Lists all registered commands with brief usage guidelines. Including the list of applicable parameters. |
| info | info | Shows firmware details and version. |
| uid | uid | Shows STM32 UID. |
| date_set | date_set <date&time> | Sets date and time of the MCU system. |
| date_get | date_get | Gets date and time of the MCU system. |
| reset | reset | Resets the MCU System. |
| start | start <"datalog", "neai_learn", or "neai_detect" > | Starts an execution phase according to its execution context, i.e. datalog, neai_learn or neai_detect. |
| stop | stop | Stops all running execution phases. |
| CS2 - PdM Specific Commands | ||
| neai_init | neai_init | (Re)initializes the AI model by forgetting any learning. Used in the beginning and / or to create a new NanoEdge AI model. |
| neai_set | neai_set <param> <val> | Sets a PdM specific parameters in an execution context. |
| neai_get | neai_get <param> | Displays the value of the parameters in the execution context. |
| neai_save | neai_save | Saves the current knowledge of the NanoEdge AI library in the flash and returns the start and destination address, as well as the size of the memory (in byte), used to save the knowledge. |
| neai_load | neai_load | Loads the saved knowledge of the NanoEdgeAI library from the flash memory. |
| CS3 - Sensor Configuration Commands | ||
| sensor_set | sensor_set <sensorID>.<subsensorID> <param> <val> | Sets the ‘value’ of a ‘parameter’ for a sensor with sensor id provided in ‘id’. |
| sensor_get | sensor_get <sensorID>.<subsensorID> <param> | Gets the ‘value’ of a ‘parameter’ for a sensor with sensor id provided in ‘id’. |
| sensor_info | sensor_info | Lists the type and ID of all supported sensors. |
| config_load | config_load | Loads the configuration parameters from the DeviceConfig.json and execution_config.json files in the root of the microSD™ card.
|
| CS4 - File System Commands | ||
| ls | ls | Lists the directory contents. |
| cd | cd <directory path> | Changes the current working directory to the provided directory path. |
| pwd | pwd | Prints the name/path of the present working directory. |
| cat | cat <file path> | Display the (text) contents of a file. |
3. Data collection
Once the user has verified that the binary has been programmed and the serial connection is already setup to the sensor node, user can start the data logging. The following section provides the guide for performing the data log.
3.1. Configure the sensor for data logging
The FP-AI-NANOEDG1 provides the capability of logging the inertial data through onboard accelerometer LSM6DSO in STM32L562E-DK, through the CLI application. The CLI interface provides the user with a capability to not just log the data with the default configurations of the sensor but also equip the user with the capability to configure the sensor by setting different values to the different variables. The configurable options for LSM6DSO include:
- Activate/deactivate the sensor or a sub-sensor,
- Set an output data rate (ODR) from the list of available options,
- Set full scale range from the list of available options.
Furthermore, the list of all the available sensors and sub-sensors can be displayed on the CLI console by entering the command:
$ sensor_info
This command will show the supported sensors with their sensor and sub-sensor IDs. The detailed information of a given supported sensor or sub-sensor can be shown by typing command:
$ sensor_get <sensor_id.sub-sensor_Id> all
Similarly, the values for any of the available configurable parameters can be set through command
$ sensor_set <sensor_id.sub-sensor_Id> <param> <val>
3.2. Logging the data
Once the sensor is configured to the wished configuration values, the data logging can be started by simply entering the command
$ start datalog
in the CLI console. While the data logging is under progress, the LCD screen on the sensor board shows the active size of the data log in kilobytes as shown in the figure above. Once the data is logged for the required time, the process can be stopped by either pressing the Esc button, or by typing the command
$ stop
in the CLI console.
With every successful data logging command, a new data folder will be created in the SD card with the name format STM32_DL_XXX, with DeviceConfig.json and LSM6DSO files. These files contain the sensor configuration at the time of start datalog command issued, and the sensor data respectively.
4. Condition monitoring using NanoEdge AI Machine Learning library
This section provides complete method to set up an application for condition monitoring using FP-AI-NANOEDG1. The steps are provided with brief details in the diagram below.
The details on how to set up the sensor board with function pack as well as data collection have been already provided in the sections above. Following sections provide the details on library generation, installation, training and testing.
4.1. Generating a condition monitoring library
The AI libraries for the condition monitoring for this function pack are generated and provided by NanoEdge AI Studio, a solution provided by Cartesiam. This section provides step-by-step guide on how to generate these condition monitoring libraries, installing them on the sensor node by linking them to the provided project in FP-AI-NANOEDG1 and then using them to perform the condition monitoring on the edge, by running first in training and then detection mode.
4.1.1. Data logging for normal and abnormal conditions
The library generation requires some sample data to be provided to the NanoEdge AI Studio. This data will provide the context of the set up to be monitored. The first step is to log the data for the normal and abnormal conditions using the data logging application provided in this function pack. Note that this data is to generate the libraries, so the users are advised to record data by exposing all the possible normal and any known abnormal conditions.
4.1.2. Data preparation for library generation with NanoEdge AI Studio
The data logged through the datalogger is in the binary format and is not user readable in its current condition. In order to convert this data to useful form, FP-AI-NANOEDG1 provides Python™ utility scripts. These scripts along with example usage can be found in a Parser_for_HS_Logged_Data.ipynb , a Python™ Jupyter™ Notebok, located on the path FP-AI-NANOEDG1/Utilities/AI-resources/DataLog/ as shown in the figure below.
4.1.3. Library generation using NanoEdge AI Studio
The process of generating the libraries with Cartesiam NanoEdge AI studio consists of five steps.
- Hardware description
- Choosing a microcontroller type : Arm® Cortex®-M33
- Maximum amount of RAM to be allocated for the library: Usually a few Kbytes will suffice (but it depends on the data frame length used in the process of data preparation).
- Sensor type : 3-axis accelerometer
- Providing the sample contextual data to adjust and gauge the performance of the chosen model. This step will require data for:
- Nominal or normal case
- Abnormal case
- Benchmarking of available models and choose the one that complies with the requirements.
- Validating the model for learning and testing through the provided emulator which emulates the behavior of the library on the edge.
- The final step is to compile and download the libraries. In this process, the flag
"-mfloat-abi"has to becheckedfor using libraries with hardware FPU.
Detailed documentation on the NanoEdge AI Studio.
4.2. Installing the NanoEdgeAI Machine Learning library
Once the libraries are generated and downloaded from Cartesiam NanoEdge AI Studio, the next step is to incorporate these libraries to FP-AI-NANOEDG1. The FP-AI-NANOEDG1, comes with the library stubs in the place of the actual libraries generated by NanoEdge AI Studio. This is done to make it easy for users to link the generated libraries and have a place holder for the libraries, which are generated through following the steps provided in the previous section. In order to link the actual libraries, the user needs to copy the generated libraries and replace the existing stub/dummy libraries and header files NanoEdgeAI.h, and libneai.a files present in the folders Inc, and lib, respectively. The relative paths of these folders are /FP_AI_NANOEDG1/Middlewares/Third_Party/Cartesiam_NanoEdge_AI_Library/ as shown in the figure below.
Once these files copied, the project must be reconstructed and programmed on the sensor board in order to link the libraries. For this, user needs to open the .project file from the FP-AI-NANOEDG1 folder, located at the path FP_AI_NANOEDG1/Projects/STM32L562E-DK/Applications/NanoEdgeConcole/STM32CubeIDE/ as shown in the step 2 in figure below.
To install the new firmware after linking the library, connect the sensor board and rebuild the project using the play button. See the console for the outputs and wait for the Build and Download success message as shown in step 4 in the figure below.
4.3. Testing the NanoEdgeAI Machine Learning library
Once the Downoad verified successfully message appears, the generated AI libraries are linked to the firmware on the sensor board and are ready to test.
Note that in their current form, the libraries have zero knowledge of the normal and abnormal conditions, and to perform the condition monitoring effectively, it is important to learn the normal behavior of the setup/machine to be modeled. This can be done by starting the learning process. The learning phase can be started by issuing start neai_learn command in console. Starting this command will show the process of learning and will show a message on the console every time learning is performed on a new signal, as shown in below code snippet.
$ start neai_learn NanoEdgeAI: starting $ 1 signal learning: SUCCESS. 2 signal learning: SUCCESS. 3 signal learning: SUCCESS. 4 signal learning: SUCCESS. 5 signal learning: SUCCESS. 6 signal learning: SUCCESS.
The process can be stopped either by issuing the
$ stop
command or by simply pressing the ESC key on the keyboard.
The alternate way to start the learning process is to long press the user button, as shown in the splash screen of the sensor node. This button can be used to start the learning in standalone as well as in computer-operated mode. The process can be stopped by simply pressing the user button again.
Once the normal conditions are learnt, the user can start the condition monitoring process by starting the neai_detect mode. This mode can be started by issuing the command
$ start neai_detect NanoEdgeAI: starting
or by short pressing the user button on the sensor board. The LCD will show the detection mode is activated along with the status through the LED in the screen, Green for Normal, Orange for Anomaly, and Red for Failure, as shown in the screen shots in figure above.
4.3.1. Some hyper-parameters in condition monitoring
For the convenience of the users CLI also provides some handy options to run the inference and learning process of the condition monitoring easily. Users can see all the variables they can play with by issuing following command:
$ neai_get all NanoEdgeAI: signals = 0 NanoEdgeAI: sensitivity = 1.000000 NanoEdgeAI: threshold = 95 NanoEdgeAI: timer = 0
Each of the these parameters can be configured using neai_set <param> <val> command. The details are provided in the command table above.
This section provides small details on neai_set signals <nr of signals> . Using this command, a user can start the learning or condition monitoring mode for a given number of samples. For example, to learn on 100 signals, the user can issue this command, before issuing the learn command. In the following snippet an example is presented where the learning is performed for 10 signals.
$ neai_set signals 10 NanoEdgeAI: signals set to 10 $ start neai_learn NanoEdgeAI: starting $ 1 signal learning: SUCCESS. 2 signal learning: SUCCESS. 3 signal learning: SUCCESS. 4 signal learning: SUCCESS. 5 signal learning: SUCCESS. 6 signal learning: SUCCESS. 7 signal learning: SUCCESS. 8 signal learning: SUCCESS. 9 signal learning: SUCCESS. 10 signal learning: SUCCESS. NanoEdge AI: stopped
The threshold parameter is used to report the anomalies. For any signal which has similarities below the threshold value is reported as anomaly. The default value used in the CLI application is 95. Users can change this value by using neai_set threshold <val> command.
The sensitivity parameter is used as an emphasis parameter. The default value is set to 1. Increasing this sensitivity will mean that the matching of the signals is to be performed more strictly, reducing it will also reduce the matching calculation process, i.e. resulting in higher matching values.
For details, users are invited to read the detailed documentation of NanoEdge AI studio.