Neural Networks support for Android is built around the Android NNAPI stack[1]. A specific support library has been developed by VeriSilicon to benefit from the NPU hardware[2]. A small STMicroelectronics integration layer has been introduced to package these components into the OpenSTDroid delivery.
1. Neural Networks overview[edit | edit source]
The Android Neural Networks API (NNAPI) provides a standard way for applications and frameworks to offload inference workloads to accelerator-specific implementations. In the Android source tree, packages/modules/NeuralNetworks contains the generic runtime, shared utilities, test infrastructure and sample integration points.
From an NDK point of view, NNAPI exposes a programming model centered on device discovery, model definition, compilation, execution, memory objects and events[1]. This is consistent with the internal organization of the support library project, which provides dedicated implementation blocks for Device, Model, Compilation, Execution, Memory and Event.
For STMicroelectronics platforms, two practical integration paths are considered:
- NN-SL approach: the compiled support library
libVsiSupportLibrary.sois integrated through an Android shell-service. - TensorFlow Lite approach: TensorFlow Lite applications or tools also use
libVsiSupportLibrary.soto offload work.
2. Android architecture[edit | edit source]
The architecture is split between a generic Android side, a VeriSilicon support library implementation side, and an STM integration side.
Some explanations about the main components:
- Android applications and frameworks: application can be based on TensorFlow Lite, or NNAPI client APIs.
- Android NNAPI module (
packages/modules/NeuralNetworks): provides the NNAPI runtime and tests. - NN-SL path: Android loads
libVsiSupportLibrary.sothrough a shell service flow. - TensorFlow Lite path: benchmark_model or a TensorFlow Lite application enables NNAPI and points to
libVsiSupportLibrary.so. - STM integration component (
device/stm/stm32mp2/hardware/neuralnetworks) provides the STM-specific integration layer. - VeriSilicon NN-SL project: it is the source integration used to build
libVsiSupportLibrary.soand connects NNAPI concepts to the vendor accelerator stack. This source project itself is not installed in OpenSTDroid; only the compiled libraries are delivered through the STM integration layer. - TIM-VX: it is the VeriSilicon inference layer used underneath the support library. It can be built with NN-SL.
- Vendor acceleration stack: relies on TIM-VX, OpenVX and NPU backend components.
2.1. NN-SL approach[edit | edit source]
The NN-SL flow focuses on Android-native integration and compliance. The VeriSilicon nn-sl project[2] describes:
- building VsiSupportLibrary with the Android NDK,
- patching
packages/modules/NeuralNetworksto build a shell service, - running VTS or CTS against the support-library-based device.
This path is the best fit when the objective is to validate a proper NNAPI support-library integration against Android test suites.
2.2. TensorFlow Lite approach[edit | edit source]
The TensorFlow Lite oriented flow is useful for quick bring-up and functional validation. The STM integration layer uses a simple execution model:
- load sources and libraries,
- build and install VSI NN related binaries,
- run benchmark_model with NNAPI enabled,
This path is simple to demonstrate because the application stays in TensorFlow Lite while NNAPI delegates execution to the support library.
2.3. Recompiling libraries[edit | edit source]
Prerequisite : The Android NDK must be installed (see on Android developer page[3] for more information)
- Download and extract the Android NDK in your desired directory (here example $HOME/Android/):
- export the ANDROID_NDK_TOOLCHAIN environment variable:
export ANDROID_NDK_TOOLCHAIN="$HOME/Android/android-ndk-r29"
The integration layer provides helper scripts to reload external sources and rebuild the prebuilt binaries delivered in the OpenSTDroid distribution.
To load the Neural Networks source packages into the distribution tree according to the STM32MP2 configuration, execute the following command:
load_nn
To build the vendor binaries with the Android NDK, execute the following command:
build_nn -i
It then performs the following actions:
- it configures and builds the VeriSilicon support library project,
- it builds TIM-VX at the same time and produces
libtim-vx.so, - it rebuilds the TensorFlow Lite benchmark_model test binary,
- when called with
-ioption, it copies the generated binaries back into the STM prebuilt integration layer.
3. How to test[edit | edit source]
Two simple commands are commonly used to validate the integration.
Following VTS test can be used:
atest VtsHalNeuralnetworksTargetTest --test-filter=TestGenerated/GeneratedTest.Test/android_hardware_neuralnetworks_IDevice_nnapi_sample_sl_updatable_reshape
For TensorFlow Lite functional validation, the following benchmark_model command can be used:
adb shell "/vendor/bin/benchmark_model --graph=/vendor/etc/models/mobilenet_v1_1_0_224_quant_tflite --use_nnapi=true --nnapi_support_library_path=/vendor/lib64/libVsiSupportLibrary.so --nnapi_accelerator_name=vsi-device-0"
4. References[edit | edit source]