This article explains how to build an example using the libcoral API[1]. The libcoral is a C/C++ API which is used for three main purposes:
- Inferencing : facilitates the implementation of neural network inference on Coral Edge TPU
- Pipelining : provides functions to pipeline a model on multiple Coral Edge TPUs
- Transfer learning : enables on board transfer learning
Several examples can be found on the libcoral GitHub[2]. This article will only show how to build a simple image classification example from scratch. The method used here can easily be applied for other examples.
1. Description[edit source]
This simple example is based on two image classification models which allows identification of the subject represented by an image.
The purpose of this example is beyond the image classification aspect to demonstrate how to use the libcoral API to realize inferences of two models alternatively on only one Google Coral Edge TPU[3]
This example depends only on the TensorFlow Lite[4] interpreter and the libraries associate to the Edge TPU.
The models used with this example are two variants, Inat_bird and Inat_plant respectively for bird and plant classification of the MobileNet v2 downloaded from the Coral GitHub testing models[5].
2. Installation[edit source]
2.1. Install from the OpenSTLinux AI package repository[edit source]
After having configured the AI OpenSTLinux package you can install the X-LINUX-AI components for this examples:
apt-get install packagegroup-x-linux-ai-tflite-edgetpu
Then restart the demo launcher: - For OpenSTLinux distribution with a version lower than 4.0 use
systemctl restart weston@root
- For other OpenSTLinux distribution use :
systemctl restart weston-launch
2.2. Source code location[edit source]
The source code and the header files of this example are located in the libcoral API examples GitHub [6].
3. How to use the example[edit source]
3.1. Installation of the X-LINUX-AI SDK[edit source]
First of all, the installation of the X-LINUX-SDK is required to be able to cross-compile AI applications for STM32 boards. To do this, follow the wiki article How to install and use the X-LINUX-AI SDK add-on[7].
3.2. Start the SDK[edit source]
Once the OpenSTLinux SDK installed, go to the installation directory and source the environment :
cd $HOME/STM32MPU_workspace/STM32MP15-Ecosystem-v4.0.0/Developer-Package/SDK
source environment-setup-cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi
3.3. Setup file tree[edit source]
This example require the following file tree to operate out of the box :
mkdir -p source/coral/example cd source/coral/example
3.4. Download the example[edit source]
As mentioned before the source and header files must be downloaded from the libcoral API examples GitHub [6] :
wget https://raw.githubusercontent.com/google-coral/libcoral/master/coral/examples/two_models_one_tpu.cc wget https://raw.githubusercontent.com/google-coral/libcoral/master/coral/examples/file_utils.cc wget https://raw.githubusercontent.com/google-coral/libcoral/master/coral/examples/file_utils.h
3.5. Create the Makefile[edit source]
The cross-compilation of the example for an STM32 board is realized through a Makefile.
This Makefile is defined as follow :
TARGET_BIN = two_models_one_tpu CXXFLAGS += -Wall $(shell pkg-config --cflags absl_base libglog) CXXFLAGS += -I../../ # Fix undefined reference during the link CXXFLAGS += -std=c++11 LDFLAGS = $(shell pkg-config --libs absl_base libglog) LDFLAGS += -lcoral -ledgetpu -ltensorflow-lite -lglog LDFLAGS += \ -labsl_flags_internal \ -labsl_flags_marshalling \ -labsl_flags_reflection \ -labsl_statusor \ -labsl_flags_parse \ -labsl_strings \ -labsl_throw_delegate SRCS = two_models_one_tpu.cc file_utils.cc OBJS = $(SRCS:.cc=.o) all: $(TARGET_BIN) $(OBJS): $(SRCS) $(CXX) $(CXXFLAGS) -c $^ $(TARGET_BIN): $(OBJS) $(CXX) $(LDFLAGS) $^ -o $@ clean: rm -rf $(OBJS) $(TARGET_BIN)
Minimal required libraries like Abseil, Glog, Tflite, Edgetpu and Coral are linked during the cross-compilation.
3.6. Download and prepare test data[edit source]
An additional package is necessary on the host PC to preprocess data for this example : imagemagick
apt-get install imagemagick
First create the directory to store test data :
mkdir edgetpu_cpp_example
Next download the models and the test pictures :
wget -O edgetpu_cpp_example/inat_plant_edgetpu.tflite https://github.com/google-coral/edgetpu/raw/master/test_data/mobilenet_v2_1.0_224_inat_plant_quant_edgetpu.tflite wget -O edgetpu_cpp_example/inat_bird_edgetpu.tflite https://github.com/google-coral/edgetpu/raw/master/test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite wget -O edgetpu_cpp_example/inat_plant_labels.txt https://github.com/google-coral/edgetpu/raw/master/test_data/inat_plant_labels.txt wget -O edgetpu_cpp_example/inat_bird_labels.txt https://github.com/google-coral/edgetpu/raw/master/test_data/inat_bird_labels.txt wget -O edgetpu_cpp_example/bird.jpg https://farm3.staticflickr.com/8008/7523974676_40bbeef7e3_o.jpg wget -O edgetpu_cpp_example/plant.jpg https://c2.staticflickr.com/1/62/184682050_db90d84573_o.jpg
Finally pre-process pictures to make them fit with the inputs shape of the models :
cd edgetpu_cpp_example && convert bird.jpg -resize 224x224! bird.rgb && convert plant.jpg -resize 224x224! plant.rgb
3.7. Cross-compilation and launch[edit source]
Run the cross-compilation :
cd .. make
Once the compilation is finished, a binary file named two_models_one_tpu has been created.
Copy the binary file and the test data directory on the board :
scp -r edgetpu_cpp_example/ root@<board_ip>:/path/ scp two_models_one_tpu root@<board_ip>:/path/
Connect to the board and launch the example :
./two_models_one_tpu
After 2000 inferences if everything went well, the result should be something like that :
Running model: edgetpu_cpp_example/inat_bird_edgetpu.tflite and model: edgetpu_cpp_example/inat_plant_edgetpu.tflite for 2000 inferences [Bird image analysis] max value index: 659 value: 0.652344 [Plant image analysis] max value index: 1680 value: 0.964844 Using one Edge TPU, # inferences: 2000 costs: 106.278 seconds.
Where the max value index represent the index of the class detected and the value represent the confidence. On these particular pictures the bird detected is a Poecile atricapillus (Black-capped Chickadee) and the plant is a Helianthus annuus (Sunflower). The index and the name of each class are available in the inat_bird_labels.txt and inat_plant_labels.txt stored in the edgetpu_cpp_example directory.
4. References[edit source]