Difference between revisions of "X-LINUX-AI - object detection using TensorFlow Lite Python runtime"

[unchecked revision] [quality revision]
m
m
 

This article explains how to experiment with TensorFlow Lite[1] applications for object detection based on the COCO SSD MobileNet v1 model using TensorFlow Lite Python runtime.

Info.png Python applications are good for prototyping but are less efficient than C/C++ applications application

1 Description[edit]

The object detection[2] neural network model allows identification and localization of a known object within an image.

Python TensorFlow Lite object detection application

The application enables OpenCV camera streaming (or test data picture) and the TensorFlow Lite[1] interpreter runing the NN inference based on the camera (or test data pictures) inputs.
It is a multi-process Python application that allows allow to run in parallel the camera preview (on the CPU core 0) and the neural network inference (using the CPU core 1) to run in parallel .
The user interface is implemented using Python GTK.

The model used with this application is the COCO SSD MobileNet v1 downloaded from the TensorFlow Lite object detection overview page[2].

2 Installation[edit]

2.1 Install from the OpenSTLinux AI package repository[edit]

Warning.png The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

After having configured the AI OpenSTLinux package you can install the X-LINUX-AI components for this application:

Board $> apt-get install tflite-cv-apps-object-detection-python

Then And restart the demo launcher:

Board $> systemctl restart weston@root

2.2 Source code location[edit]

The objdetect_tfl_multiprocessing.py Python script is available:

  • in the Openembedded OpenSTLinux Distribution distribution with X-LINUX-AI Expansion Packageexpansion package:
<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py
  • on the target:
/usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py
  • on GitHub:
https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/dunfell/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py

3 How to use the applicationit[edit]

3.1 Launching Launch application via the demo launcher[edit]

Launch python tfl object detection.png

3.2 Executing Or execute with the command line[edit]

The Python script objdetect_tfl_multiprocessing.py application is located in the userfs partition:

/usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py

It accepts the following input parameters:

usage: objdetect_tfl_multiprocessing.py [-h] [-i IMAGE] [-v VIDEO_DEVICE] [--frame_width FRAME_WIDTH] [--frame_height FRAME_HEIGHT]   
                                        [--framerate FRAMERATE] [-m MODEL_FILE] [-l LABEL_FILE] [--input_mean INPUT_MEAN]             
                                        [--input_std INPUT_STD]                                                                       
                                                                                                                                      
optional arguments:                                                                                                                   
  -h, --help            show this help message and exit                                                                               
  -i IMAGE, --image IMAGE                                                                                                             
                        image directory with image to be classified                                                                   
  -v VIDEO_DEVICE, --video_device VIDEO_DEVICE                                                                                        
                        video device (default /dev/video0)                                                                            
  --frame_width FRAME_WIDTH                                                                                                           
                        width of the camera frame (default is 320)                                                                    
  --frame_height FRAME_HEIGHT                                                                                                         
                        height of the camera frame (default is 240)                                                                   
  --framerate FRAMERATE                                                                                                               
                        framerate of the camera (default is 15fps)                                                                    
  -m MODEL_FILE, --model_file MODEL_FILE                                                                                              
                        .tflite model to be executed                                                                                  
  -l LABEL_FILE, --label_file LABEL_FILE                                                                                              
                        name of file containing labels                                                                                
  --input_mean INPUT_MEAN                                                                                                             
                        input mean                                                                                                    
  --input_std INPUT_STD                                                                                                               
                        input standard deviation 

3.3 Testing with COCO ssd MobileNet v1[edit]

The model used for test is the detect.tflite downloaded from object detection overview[2]

Info.png The different objects the neural network is able to detect are listed in the labels.txt file located in the target:

/usr/local/demo-ai/computer-vision/models/coco_ssd_mobilenet/labels.txt


To ease launching of the Python script, two shell scripts are available:

  • launch object detection based on camera frame inputs
Board $> /usr/local/demo-ai/computer-vision/tflite-object-detection/python/launch_python_objdetect_tfl_coco_ssd_mobilenet.sh
  • launch object detection based on the pictures located in /usr/local/demo-ai/computer-vision/models/coco_ssd_mobilenet/testdata directory
Board $>  /usr/local/demo-ai/computer-vision/tflite-object-detection/python/launch_python_objdetect_tfl_coco_ssd_mobilenet_testdata.sh
Info.png Note that you need to populate the testdata directory with your own data sets.

The pictures are then randomly read from the testdata directory

4 References[edit]


This article explains how to experiment with {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url>[https://www.tensorflow.org/lite TensorFlow Lite]</ref>}} applications for object detection based on the COCO SSD MobileNet v1 model using TensorFlow Lite Python runtime.

{{Info|Python applications are good for prototyping but are less efficient than C/C++ applicationsapplication}}

==Description==
The '''object detection'''<ref name=tflite_obj_detect_url>[https://www.tensorflow.org/lite/models/object_detection/overview TFLite object detection overview]</ref> neural network model allows identification and localization of a known object within an image.

[[File: python_tfl_object_detection_application_screenshot.png|thumb|upright=2|center|link=|Python TensorFlow Lite object detection application]]

The application enables OpenCV camera streaming (or test data picture) and the {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url></ref>}} interpreter runing the NN inference based on the camera (or test data pictures) inputs.<br>

It is a multi-process Python application that allows allow to run in parallel the camera preview (on the CPU core 0) and the neural network inference (using the CPU core 1)  to run in parallel .<br>

The user interface is implemented using Python GTK.<br>


The model used with this application is the {{Highlight|COCO SSD MobileNet v1}} downloaded from the '''TensorFlow Lite object detection overview''' page<ref name=tflite_obj_detect_url>[https://www.tensorflow.org/lite/models/object_detection/overview TFLite object detection overview]</ref>.

==Installation==
===Install from the OpenSTLinux AI package repository===
{{Warning|{{SoftwareLicenseAgreement | distribution=X-LINUX-AI}}}}
After having [[X-LINUX-AI_OpenSTLinux_Expansion_Package#Configure the AI OpenSTLinux package repository|configured the AI OpenSTLinux package]] you can install the X-LINUX-AI components for this application:
 {{Board$}} apt-get install tflite-cv-apps-object-detection-pythonThenAnd restart the demo launcher:
 {{Board$}} systemctl restart weston@root

===Source code location===
The '''objdetect_tfl_multiprocessing.py''' Python script is available:
* in the Openembedded OpenSTLinux Distributiondistribution with X-LINUX-AI Expansion Packageexpansion package:
:'''<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py'''
* on the target:
:'''/usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py'''
* on GitHub: 
:https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/dunfell/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py

==How to use the application==
===Launching it==
===Launch application via the demo launcher===
[[File: launch_python_tfl_object_detection.png|thumb|upright=2|center|link=]]

===Executing Or execute with the command line===
The Python script '''objdetect_tfl_multiprocessing.py''' application is located in the userfs partition:
 /usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py
It accepts the following input parameters:
<pre>

usage: objdetect_tfl_multiprocessing.py [-h] [-i IMAGE] [-v VIDEO_DEVICE] [--frame_width FRAME_WIDTH] [--frame_height FRAME_HEIGHT]   
                                        [--framerate FRAMERATE] [-m MODEL_FILE] [-l LABEL_FILE] [--input_mean INPUT_MEAN]             
                                        [--input_std INPUT_STD]                                                                       

optional arguments:                                                                                                                   
  -h, --help            show this help message and exit                                                                               
  -i IMAGE, --image IMAGE                                                                                                             
                        image directory with image to be classified                                                                   
  -v VIDEO_DEVICE, --video_device VIDEO_DEVICE                                                                                        
                        video device (default /dev/video0)                                                                            
  --frame_width FRAME_WIDTH                                                                                                           
                        width of the camera frame (default is 320)                                                                    
  --frame_height FRAME_HEIGHT                                                                                                         
                        height of the camera frame (default is 240)                                                                   
  --framerate FRAMERATE                                                                                                               
                        framerate of the camera (default is 15fps)                                                                    
  -m MODEL_FILE, --model_file MODEL_FILE                                                                                              
                        .tflite model to be executed                                                                                  
  -l LABEL_FILE, --label_file LABEL_FILE                                                                                              
                        name of file containing labels                                                                                
  --input_mean INPUT_MEAN                                                                                                             
                        input mean                                                                                                    
  --input_std INPUT_STD                                                                                                               
                        input standard deviation </pre>


===Testing with COCO ssd MobileNet v1===
The model used for test is the {{Highlight|detect.tflite}} downloaded from '''object detection overview'''<ref name=tflite_obj_detect_url>[https://www.tensorflow.org/lite/models/object_detection/overview TFLite object detection overview]</ref>


{{Info|The different objects the neural network is able to detect are listed in the labels.txt file located in the target:
/usr/local/demo-ai/computer-vision/models/coco_ssd_mobilenet/labels.txt}}

To ease launching of the Python script, two shell scripts are available:
* launch object detection based on camera frame inputs
 {{Board$}} /usr/local/demo-ai/computer-vision/tflite-object-detection/python/launch_python_objdetect_tfl_coco_ssd_mobilenet.sh

*launch object detection based on the pictures located in '''/usr/local/demo-ai/computer-vision/models/coco_ssd_mobilenet/testdata''' directory
 {{Board$}}  /usr/local/demo-ai/computer-vision/tflite-object-detection/python/launch_python_objdetect_tfl_coco_ssd_mobilenet_testdata.sh

{{Info|Note that you need to populate the testdata directory with your own data sets.<br>

The pictures are then randomly read from the testdata directory}}

==References==<references />

<noinclude>

[[Category:Artificial intelligence sample apps|16]]
{{PublicationRequestId | 16541 | 24Jun'20}}</noinclude>
Line 1: Line 1:
This article explains how to experiment {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url>[https://www.tensorflow.org/lite TensorFlow Lite]</ref>}} applications for object detection based on COCO SSD MobileNet v1 model using TensorFlow Lite Python runtime.
+
This article explains how to experiment with {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url>[https://www.tensorflow.org/lite TensorFlow Lite]</ref>}} applications for object detection based on the COCO SSD MobileNet v1 model using TensorFlow Lite Python runtime.
   
{{Info|Python applications are good for prototyping but are less efficient than C/C++ application}}
+
{{Info|Python applications are good for prototyping but are less efficient than C/C++ applications}}
   
 
==Description==
 
==Description==
Line 9: Line 9:
   
 
The application enables OpenCV camera streaming (or test data picture) and the {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url></ref>}} interpreter runing the NN inference based on the camera (or test data pictures) inputs.<br>
 
The application enables OpenCV camera streaming (or test data picture) and the {{Highlight|TensorFlow Lite<ref name=tensorflowlite_url></ref>}} interpreter runing the NN inference based on the camera (or test data pictures) inputs.<br>
It is a multi-process Python application that allow to run in parallel the camera preview (on the CPU core 0) and the neural network inference (using the CPU core 1).<br>
+
It is a multi-process Python application that allows the camera preview (on the CPU core 0) and the neural network inference (using the CPU core 1) to run in parallel .<br>
 
The user interface is implemented using Python GTK.<br>
 
The user interface is implemented using Python GTK.<br>
   
Line 19: Line 19:
 
After having [[X-LINUX-AI_OpenSTLinux_Expansion_Package#Configure the AI OpenSTLinux package repository|configured the AI OpenSTLinux package]] you can install the X-LINUX-AI components for this application:
 
After having [[X-LINUX-AI_OpenSTLinux_Expansion_Package#Configure the AI OpenSTLinux package repository|configured the AI OpenSTLinux package]] you can install the X-LINUX-AI components for this application:
 
  {{Board$}} apt-get install tflite-cv-apps-object-detection-python
 
  {{Board$}} apt-get install tflite-cv-apps-object-detection-python
And restart the demo launcher:
+
Then restart the demo launcher:
 
  {{Board$}} systemctl restart weston@root
 
  {{Board$}} systemctl restart weston@root
   
 
===Source code location===
 
===Source code location===
 
The '''objdetect_tfl_multiprocessing.py''' Python script is available:
 
The '''objdetect_tfl_multiprocessing.py''' Python script is available:
* in the Openembedded OpenSTLinux distribution with X-LINUX-AI expansion package:
+
* in the Openembedded OpenSTLinux Distribution with X-LINUX-AI Expansion Package:
 
:'''<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py'''
 
:'''<Distribution Package installation directory>/layers/meta-st/meta-st-stm32mpu-ai/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py'''
 
* on the target:
 
* on the target:
Line 31: Line 31:
 
:https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/dunfell/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py
 
:https://github.com/STMicroelectronics/meta-st-stm32mpu-ai/tree/dunfell/recipes-samples/tflite-cv-apps/files/object-detection/python/objdetect_tfl_multiprocessing.py
   
==How to use it==
+
==How to use the application==
===Launch application via the demo launcher===
+
===Launching via the demo launcher===
 
[[File: launch_python_tfl_object_detection.png|thumb|upright=2|center|link=]]
 
[[File: launch_python_tfl_object_detection.png|thumb|upright=2|center|link=]]
   
===Or execute with command line===
+
===Executing with the command line===
 
The Python script '''objdetect_tfl_multiprocessing.py''' application is located in the userfs partition:
 
The Python script '''objdetect_tfl_multiprocessing.py''' application is located in the userfs partition:
 
  /usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py
 
  /usr/local/demo-ai/computer-vision/tflite-object-detection/python/objdetect_tfl_multiprocessing.py