Linux remoteproc framework overview

Applicable for

This article gives information about the Linux^® remoteproc framework.

1. Framework purpose[edit source]

The remote processor (RPROC) framework allows the different platforms/architectures to control (power on, load firmware, power off) remote processors while abstracting the hardware differences. In addition it offers services to monitor and debug the remote coprocessor.

2. System overview[edit source]

2.1. Component description[edit source]

remoteproc: this is the remote processor framework generic part. Its role is to :

Load a firmware in the remote processor memory.
Parse the firmware resource table to set associated resources (such as IPC, memory carveout and traces).
Control the remote processor execution (start, stop...).
Provide a service to monitor and debug the remote firmware.

stm32_rproc: this is the remote processor platform driver. Its role is to:

Register the vendor specific functions (callback) to the RPROC framework.
Handle the platform resources associated to the remote processor (such as registers, watchdogs, reset, clock and memories).
Forward notifications (kicks) to the remote processor through the mailbox framework.

TEE remoteproc: this remote driver allow to to communicate with the OP-TEE in the secure context to authenticate and load a signed firmware image. for more detail on the feature refer to How to protect the coprocessor firmware article.

2.2. Firmware image format supported[edit source]

The remoteproc framework support 2 formas that are exclusive:

ELF format: The ELF format is supported by the Linux remoteproc framework
TEE format: the TEE format is supported by the OP-TEE remoteproc framework

2.3. API description[edit source]

The API usage and remote processor binary firmware structure (resource table, ...) are described in the Linux kernel remoteproc documentation ^[1].

3. Configuration[edit source]

Warning

The remoteproc framework needs the mailbox framework to be configured. Refer to mailbox kernel configuration for details.

3.1. Kernel configuration[edit source]

Activate the remoteproc driver and framework in the kernel configuration using the Linux Menuconfig tool: Menuconfig or how to configure kernel.

Device drivers  --->
   Remoteproc drivers  --->
     <*> Support for Remote Processor subsystem
     <*> STM32 remoteproc support

to support the TEE image the TEE remoteproc driver module also be enable

     <*> Remoteproc support by a TEE application

3.2. Device tree configuration[edit source]

The STM32 remoteproc bindings^[2] documentation deals with all required or optional STM32 remoteproc DT properties.

It also introduces memory regions properties that define the memory base addresses and sizes, used by the Cortex-M or shared with it. These memory regions are described in the device tree from the Arm^® Cortex^®-A point of view.
Simplified example:

/* Memory region declaration, containing vring and rpmsg buffers */
reserved-memory {
	/* memory region reserved for the Cortex-M firmware code and data */
fw_mem: retram@0x38000000
	{
		reg = <0x38000000 0x10000>;
	};
	/* memory region reserved for inter processor communication */
	ipc_mem: mcusram@0x10000000 {
		reg = <0x10000000 0x40000>;
	};

 /* stm32 remoteproc device */
 cm_rproc: cm@0 {
	...
	memory-region = <&fw_mem>, <&ipc_mem>;
	...
};

3.2.1. STM32MP15x lines [edit source]

The memory regions properties define the RETRAM and MCU SRAMs base addresses and sizes in RETRAM and MCU SRAMs reserved for the Cortex-M firmware and for inter processor communication.

The Memory regions are defined at board level:

on the STM32MP157Fx-ED1 Evaluation board: The memory-regions are declared and referenced in stm32mp157f-ed1.dts^[3],
on the STM32MP157Fx-DK Discovery board: The memory-regions are declared and referenced in stm32mp15xx-dkx.dtsi^[4].

Device tree deltas between signed and non signed firmware support:

Non-signed (ELF) firmware
image support

Signed firmware
image support

Comment

&m4_rproc {
	compatible = "st,stm32mp1-m4";
	/*
	 * Declare  memories used to run the CM4 firmware and shared memory
	 * used for inter-processors communication (including the resource table)
	 */
	memory-region =  <&retram>, <&mcuram>, <&mcuram2>,
				    <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>;
	/* The resets are managed by the Linux  remoteproc driver */
	resets = <&scmi_reset RST_SCMI_MCU>,
			 <&scmi_reset RST_SCMI_MCU_HOLD_BOOT>;
	reset-names = "mcu_rst", "hold_boot";
};

&m4_rproc {
	compatible = "st,stm32mp1-m4-tee";
	/*
	 * Declare only shared memory used for inter-processors communication
	 * (including the resource table)
	 */
	
	/* No reset declaration, the resets are managed by the OP-TEE remoteproc driver */



};

Information

The firmware memory mapping must be set according to these values in the STM32Cube firmware linker script.

For additional details, please refer to STM32MP15 Memory mapping.

3.2.2. STM32MP25x lines [edit source]

The memory regions properties define the DDR, RETRAM and/or MCU SRAMs base addresses and sizes reserved for the Cortex-M secure firmware (TF-M) and non-secure firmware (STM32Cube firmware), and reserved for inter processor communication.

The Memory regions are defined at board level:

On the STM32MP257x-EV1 Evaluation board:
- The memory-regions are declared in stm32mp257f-ev1-ca35tdcid-resmem.dtsi^[5]
- The memory-regions are referenced in the m33_rproc node in stm32mp257f-ev1.dts^[6]
On the STM32MP257x-DK Discovery board:
- The memory-regions are declared in stm32mp257f-dk-ca35tdcid-resmem.dtsi^[7]
- The memory-regions are referenced in the m33_rproc node in stm32mp257f-dk.dts^[8]

Device tree deltas between signed and non signed firmware support:

Non-signed (ELF) firmware
image support

Signed firmware
image support

Comment

&m33_rproc {
	compatible = "st,stm32mp2-m33";
	/*
	 * Declare  memories used to run the CM33 firmware and shared memory
	 * used for inter-processors communication (including the resource table)
	 */
	memory-region = <&ipc_shmem_1>, <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>,
	                 <&cm33_cube_fw>, <&cm33_cube_data>;
	/* The resets are managed by the Linux  remoteproc driver */
	resets = <&scmi_reset RST_SCMI_C2_R>,
			 <&scmi_reset RST_SCMI_C2_HOLDBOOT_R>;
	reset-names = "mcu_rst", "hold_boot";
   /* declare boot address register */
	st,syscfg-nsvtor = <&a35ss_syscfg 0xa8 0xffffff80>;
};

&m33_rproc {
	compatible = "st,stm32mp2-m33-tee";
	/*
	 * Declare only shared memory used for inter-processors communication
	 * (including the resource table)
	 */
	memory-region = <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>, <&ipc_shmem_1>;
	
	/* No reset declaration, the resets are managed by the OP-TEE remoteproc driver */



	/* No boot address register declaration */

};

{{Info| The firmware memory mapping must be set according to these values in the:

STM32Cube firmware linker scripts.
TF-M linker script
OP-TEE RIF device tree.

For additional details, please refer to Memory mapping.

4. How to use the framework[edit source]

4.1. Remote processor boot[edit source]

There are three possibilities to load and start the remote processor firmware:

Start the firmware through the sysFS interface.
Automatically start the firmware on remoteproc driver probing (not recommended by STMicroelectronics).
Early boot the firmware during boot time (before Linux boot).

4.1.1. Remote processor boot through sysfs[edit source]

The firmware components are stored in the file system, by default in the /lib/firmware/ folder. Optionally another location can be set. In this case the remoteproc framework parses this new path in priority.

Below the command for adding a new path for firmware parsing:

Board $> echo -n <firmware_path> > /sys/module/firmware_class/parameters/path

Warning

This path is common for all firmwares loaded by Linux (Bluetooth, Wifi...)

The supported image format can be check using command:

Board $> cat sys/class/remoteproc/remoteproc0/fw_format

* ELF: support of non-signed image in elf format

* TEE: support of signed firmware (refer to How to protect the coprocessor firmware for details).

If the firmware elf filename differs from the default one (rproc-%s-fw), set the name with the following command: ( replace X with remoteproc instance number: 0 by default)

Board $> echo -n <firmware_name> > /sys/class/remoteproc/remoteprocX/firmware

To start the firmware, use the following command:

Board $> echo start >/sys/class/remoteproc/remoteprocX/state

Information

Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.

4.1.2. Remote processor 'auto' boot (not recommended by STMicroelectronics)[edit source]

The remote processor can be automatically booted during platform boot. To do this, the following conditions must be fulfilled:

The firmware must be present in /lib/firmware before the remoteproc driver is probed.
The filesystem on Linux (Cortex-A) must be available before the remoteproc driver is probed. However, in normal conditions, the remoteproc driver is probed before the filesystem is mounted, and the firmware is consequently not available during the Linux driver probing phase. Possible solutions could be:

to use an initramfs^[9]
or compile remoteproc as a module and not as kernel built-in driver.

The firmware must be named rproc-%s-fw, where %s corresponds to the name of the remoteproc node in the device tree. For example, for rproc-m4-fw, the remoteproc device tree must be defined as follows:

m4 {
	compatible = "st,stm32mp1-rproc";
[...]
	status = "okay";
};

The "st,auto_boot" property has to be defined in the remoteproc node device tree:

m4_rproc: m4@0  {
	compatible = "st,stm32mp1-rproc";
[...]
        st,auto-boot =  <1>;
        status = "okay";
};

4.1.3. Remote processor 'early' boot[edit source]

The coprocessor can be started by the second stage bootloader (eg U-Boot). This mode allows to start the coprocessor firmware before the Linux one. For instance, it can be used to execute first actions for projects that have hard constraints on boot time.
In such case the remoteproc framework can attach itself to the firmware by parsing the resource table, based on the information added by the bootloader in the backup registers (Cortex-M state and resource table address).

4.1.3.1. Automatic attach on Linux boot[edit source]

On Linux boot, the remoteproc framework can automatically attach itself to the coprocessor firmware if the "st,auto_boot" property is defined in the remoteproc node device tree:

m4_rproc: m4@0  {
	compatible = "st,stm32mp1-rproc";
[...]
        st,auto-boot =  <1>;
        status = "okay";
};

This mode is recommended to start the resource manager framework during the Linux boot. It prevents a system resource, used by the Cortex-M only, from being disabled by Linux at the end of its boot phase.

4.1.3.2. Manual attach by Linux application[edit source]

After the Linux boot, the remoteproc framework can attach to the coprocessor firmware using the following command:

Board $> echo start >/sys/class/remoteproc/remoteprocX/state

Refer to How to start the coprocessor from the bootloader for details on this mode.

4.2. Remote processor stop[edit source]

It is possible to stop the remote processor firmware through the SysFS interface. On stop request, the stm32_rproc driver:

informs the remote firmware relying on the "shutdown" channel of the the IPCC mailbox. This mechanism allows the remote processor firmware to shut down properly.
resets the coprocessor, on "shutdown" message acknowledgement or after a timeout of 500 ms.

Information

The use of the IPCC "shutdown" channel is optional. If the mailbox channel is not declared in the device tree, the remote processor is immediately reset, without informing firmware of the remote processor.

To stop the firmware, use the following command:

 echo stop >/sys/class/remoteproc/remoteprocX/state

Note that in STM32MPU Embedded Software distribution A demonstration firmware is loaded by default.

The load service can be stopped stopped with the command:

for STM32MP15x lines

 systemctl stop st-m4firmware-load.service

for STM32MP25x lines

 systemctl stop st-m33firmware-load.service

The load service can be disbabled with the command:

for STM32MP15x lines

 systemctl disable st-m4firmware-load.service

for STM32MP25x lines

 systemctl disable st-m33firmware-load.service

5. How to trace and debug the framework[edit source]

5.1. How to monitor[edit source]

The remoteproc firmware state can be monitored using following command:

Board $> cat /sys/class/remoteproc/remoteprocX/state

5.2. How to trace[edit source]

remoteproc framework and driver debug traces can be added in the kernel logs thanks to the dynamic debug mechanism:

Board $> echo -n 'file stm32_rproc.c +p' > /sys/kernel/debug/dynamic_debug/control
Board $> echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control

A log buffer can de defined in the remoteproc firmware and declared in the resource table. If the feature is activated on the remote firmware, log traces can be dumped from the trace buffer using the following command:

Board $>cat /sys/kernel/debug/remoteproc/remoteprocX/trace0

6. References[edit source]

[1] Linux kernel remoteproc documentation

[STM32_remoteproc_bindings-2] Documentation/devicetree/bindings/remoteproc/st,stm32-rproc.yaml , Linux Foundation, STM32 remoteproc DT bindings

[stm32mp157f-ed1.dts-3] STM32MP157Fx ED1 board device tree file

[stm32mp15xx-dkx.dtsi-4] STM32MP157Fx DK board device tree file

[stm32mp257f-ev1-ca35tdcid-resmem.dtsi-5] STM32MP257F EV1 board reserved memory device tree configuration file

[stm32mp257f-ev1.dts-6] STM32MP257F EV1 board device tree configuration file

[stm32mp257f-dk-ca35tdcid-resmem.dtsi-7] STM32MP257F DK board reserved memory device tree configuration file

[stm32mp257f-dk.dts-8] STM32MP257F DK board device tree configuration file

[9] ramfs-rootfs-initramfs Linux documentation

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]