Linux remoteproc framework overview

Applicable for

STM32MP15x lines, STM32MP21x lines, STM32MP23x lines, STM32MP25x lines

---

This article gives information about the Linux^® remoteproc framework.

1. Framework purpose

The remote processor (RPROC) framework allows different platforms/architectures to control (power on, load firmware, power off) remote processors while abstracting hardware differences. In addition, it offers services to monitor and debug the remote processor.

2. System overview

Information

The stm32_m0_rproc driver is specifically designed for the Cortex®-M0+ for use with STM32MP25x lines.

2.1. Component description

• remoteproc: The remote processor framework generic part. Its role is to:
  • Load firmware in the remote processor memory.
  • Parse the firmware resource table to set associated resources (such as IPC, memory carveout, and traces).
  • Control the remote processor execution (start, stop, etc.).
  • Provide a service to monitor and debug the remote firmware.

• stm32_rproc and stm32_m0_rproc: These are the remote processor platform drivers. Their role is to:
  • Register the vendor-specific functions (callbacks) to the RPROC framework.
  • Handle the platform resources associated with the remote processor (such as registers, watchdogs, reset, clock, and memories).
  • Forward notifications (kicks) to the remote processor through the mailbox framework.

Information

The stm32_m0_rproc driver is specifically designed for the Cortex®-M0+ for use with STM32MP25x lines

• TEE remoteproc: This remote driver enables communication with OP-TEE in the secure context to authenticate and load signed firmware images. If firmware authentication is not required, this driver can be disabled. For more details, refer to the How to protect the Cortex-M coprocessor firmware article.

2.2. Firmware image format supported

The remoteproc framework supports two exclusive formats:

• ELF format: Supported by the Linux remoteproc framework.
• TEE format: Supported by the OP-TEE remoteproc framework.

2.3. API description

The API usage and remote processor binary firmware structure (resource table, etc.) are described in the Linux kernel remoteproc documentation.

3. Configuration

Warning

The remoteproc framework needs the mailbox framework to be configured. Refer to mailbox kernel configuration for details.

3.1. Kernel configuration

Activate the remoteproc driver and framework in the kernel configuration using the Linux Menuconfig tool: Menuconfig or how to configure kernel.

Device drivers  --->
   Remoteproc drivers  --->
     <*> Support for Remote Processor subsystem
     <*> STM32 remoteproc support

To support the TEE image, the TEE remoteproc driver module must also be enabled:

     <*> Remoteproc support by a TEE application

3.2. Device tree configuration

The STM32 remoteproc bindings documentation covers all required or optional STM32 remoteproc DT properties.

It also introduces memory regions properties that define the memory base addresses and sizes, used by the Cortex-M or shared with it. These memory regions are described in the device tree from the Arm^® Cortex^®-A point of view.

3.2.1. STM32MP15x lines 

The memory regions properties define the RETRAM and MCU SRAMs base addresses and sizes in RETRAM and MCU SRAMs reserved for the Cortex-M firmware and for inter-processor communication.

The memory regions are defined at board level:

• On the STM32MP157Fx-ED1 Evaluation board: The memory-regions are declared and referenced in stm32mp157f-ed1.dts
• On the STM32MP157Fx-DK Discovery board: The memory-regions are declared and referenced in stm32mp15xx-dkx.dtsi

Device tree deltas between signed and non-signed firmware support:

Non-signed (ELF) firmware image support	Signed firmware image support	Comment
&m4_rproc { compatible = "st,stm32mp1-m4"; /* * Declare memories used to run the CM4 firmware and shared memory * used for inter-processors communication (including the resource table) */ memory-region = <&retram>, <&mcuram>, <&mcuram2>, <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>; /* The resets are managed by the Linux remoteproc driver */ resets = <&scmi_reset RST_SCMI_MCU>, <&scmi_reset RST_SCMI_MCU_HOLD_BOOT>; reset-names = "mcu_rst", "hold_boot"; };	&m4_rproc { compatible = "st,stm32mp1-m4-tee"; /* * Declare only shared memory used for inter-processors communication * (including the resource table). */ memory-region = <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>, <&mcu_rsc_table>; /* No reset declaration, the resets are managed by the OP-TEE remoteproc driver */ /delete-property/ resets; /delete-property/ reset-names; };

Information

The firmware memory mapping must be set according to these values in the STM32Cube firmware linker script. For additional details, refer to STM32MP15 Memory mapping.

3.2.2. STM32MP2 series

3.2.2.1. Cortex^®-M33

The memory regions properties define the DDR, RETRAM and/or MCU SRAMs base addresses and sizes reserved for the Cortex-M secure firmware (TF-M) and non-secure firmware (STM32Cube firmware), and reserved for inter-processor communication.

The memory regions are defined at board level:

• In the kernel device tree board for non-signed firmware for A35-TD flavor :
  • The memory-regions are declared in the <board>-resmem.dtsi board reserved memory device tree configuration file
  • The m33_rproc node is defined in the <board>.dts board device tree configuration file

 &m33_rproc {
 	compatible = "st,stm32mp2-m33";
 	/*
 	 * Declare  memories used to run the CM33 firmware and shared memory
 	 * used for inter-processors communication (including the resource table)
 	 */
 	memory-region = <&ipc_shmem_1>, <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>,
 	                <&cm33_cube_fw>, <&cm33_cube_data>;
 	/* The resets are managed by the Linux  remoteproc driver */
 	resets = <&scmi_reset RST_SCMI_C2_R>, <&scmi_reset RST_SCMI_C2_HOLDBOOT_R>;
 	reset-names = "mcu_rst", "hold_boot";
    /* Declare boot address register */
 	st,syscfg-nsvtor = <&a35ss_syscfg 0xa8 0xffffff80>;
 };

• In the External device tree for signed firmware for A35-TD flavor :
  • The memory-regions are declared in the <board>ca35tdcid-resmem.dtsi board reserved memory device tree configuration file
  • The m33_rproc node is defined in the <board>-ca35tdcid-ostl.dts board device tree configuration file

 &m33_rproc {
 	compatible = "st,stm32mp2-m33-tee";
 	/*
 	 * Declare only shared memory used for inter-processors communication
 	 * (including the resource table)
 	 */
 	memory-region = <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>, <&ipc_shmem_1>;
 	
 	/* No reset declaration, the resets are managed by the OP-TEE remoteproc driver */
    /delete-property/ resets;
    /delete-property/ reset-names;
  	/* No boot address register declaration */
 
 };

• In the External device tree for M33-TD flavor :
  • The memory-regions are declared in the <board>m33tdcid-resmem.dtsi board reserved memory device tree configuration file
  • The m33_rproc node is defined in the <board>-m33tdcid-ostl.dts board device tree configuration file

 &m33_rproc {
 	compatible = "st,stm32mp2-m33";
 	/*
 	 * Declare only shared memory used for inter-processors communication
 	 * (including the resource table)
 	 */
 	memory-region = <&vdev0vring0>, <&vdev0vring1>, <&vdev0buffer>, <&ipc_shmem_1>;
 	
 	/* No reset declaration, the Cortex-A35 just "attach" to the Cortex-M33 for IPC */
    /delete-property/ resets;
    /delete-property/ reset-names;
  	/* No boot address register declaration */
 
 };

Information

The firmware memory mapping must be set according to these values in the: STM32Cube firmware linker scripts. TF-M linker script OP-TEE resource memory device tree. For additional details, refer to Memory mapping.

3.2.2.1.1. Boot address (A35-TD flavor only)

The Cortex-M33 boots using the non-secure vector table base address.

• Non-Signed (ELF) Firmware Image Support:

The Linux kernel remoteproc determines the boot address based on the .isr_vectors linker section's base address. This section must be defined in the STM32Cube firmware linker scripts and should contain the vector table.

• Signed Firmware Image Support:

When the firmware image is signed, the boot address is provided as part of the signing process. For additional details, refer to How to Protect the Cortex-M processor Firmware.

3.2.2.1.2. Power domain selection (A35-TD flavor only)

It is possible to select the power domain when the Cortex-A35 requests to enter low-power mode while the Cortex-M33 remains active. The device tree property keep-power-in-suspend determines the power domain used during suspend.

• If keep-power-in-suspend is defined, the Cortex-M33 is attached to the "sleep" power domain.
• If it is not defined, the Cortex-M33 is attached to the "default" power domain.

 &m33_rproc {
 	compatible = "st,stm32mp2-m33-tee";
 	power-domains = <&cluster_pd>, <&ret_pd>;
 	power-domain-names = "default", "sleep";
    keep-power-in-suspend;   /* when defined the remote processor is attached to the "sleep" power domain */
 };

This mechanism allows, for example, the management of the DDR's power state depending on whether the Cortex-M33 is running in DDR or internal RAMs.

For additional details, refer to [Power overview].

3.2.2.2. Cortex^®-M0+ (STM32MP25x lines  only)

The memory regions properties define the LPSRAMs base addresses and sizes reserved for the Cortex^®-M0+ non-secure firmware (STM32Cube firmware), and reserved for inter-processor communication.

The memory regions are defined at board level:

• On the STM32MP257x-EV1 Evaluation board:
  • The memory-regions are declared in stm32mp257f-ev1-ca35tdcid-resmem.dtsi
  • The memory-regions are referenced in the m0_rproc node in stm32mp257f-ev1.dts
• On the STM32MP257x-DK Discovery board:
  • The memory-regions are declared in stm32mp257f-dk-ca35tdcid-resmem.dtsi
  • The memory-regions are referenced in the m0_rproc node in stm32mp257f-dk.dts

 &m0_rproc {
 	compatible = "st,stm32mp2-m0";
 	/*
 	 * Declare  memories used to run the CM0+ firmware and shared memory
 	 * used for inter-processors communication
 	 */
 	memory-region = <&ipc_shmem_2>, <&cm0_cube_fw>, <&cm0_cube_data>;
 	/* The resets are managed by the Linux remoteproc driver */
 	resets = <&rcc C3_R>;
 	reset-names = "mcu_rst";
 };

Information

The firmware memory mapping must be set according to these values in the: STM32Cube firmware linker scripts. OP-TEE resource memory device tree. For additional details, refer to Memory mapping.

4. How to use the framework

4.1. Remote processor boot

• STM32MP15x lines  and STM32MP2 series A35-TD flavor

There are three possibilities to interact with the remote processor:

• Load the remote processor firmware and start it through the sysFS interface.
• Automatically load the remote processor firmware and start it on remoteproc driver probing (auto boot).
• Attach to the remote processor early booted (before Linux boot).

• STM32MP2 series M33-TD flavor

Only attachment to the remote processor is supported, as the Cortex-M is early booted.

4.1.1. Remote processor boot through sysfs

• The firmware components are stored in the file system, by default in the /lib/firmware/ folder. Optionally, another location can be set. In this case, the remoteproc framework parses this new path in priority.

Below is the command for adding a new path for firmware parsing:

 echo -n <firmware_path> > /sys/module/firmware_class/parameters/path

Warning

This path is common for all firmwares loaded by Linux (Bluetooth, Wifi, etc.)

• The supported image format can be checked using the command:

 cat /sys/class/remoteproc/remoteprocX/fw_format

• ELF: support of non-signed image in ELF format
• TEE: support of signed firmware (refer to How to protect the Cortex-M coprocessor firmware for details).

• If the firmware ELF filename differs from the default one (rproc-%s-fw), set the name with the following command (replace X with the remoteproc instance number, 0 by default):

 echo -n <firmware_name> > /sys/class/remoteproc/remoteprocX/firmware

• To start the firmware, use the following command:

 echo start > /sys/class/remoteproc/remoteprocX/state

Information

Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.

4.1.2. Remote processor 'auto' boot

The remote processor can be automatically booted during platform boot. To do this, the following conditions must be fulfilled:

• The firmware must be present in /lib/firmware before the remoteproc driver is probed.
• The filesystem on Linux (Cortex-A) must be available before the remoteproc driver is probed. However, in normal conditions, the remoteproc driver is probed before the filesystem is mounted, and the firmware is consequently not available during the Linux driver probing phase. Possible solutions include:
  • Using an initramfs
  • Compiling remoteproc as a module and not as a kernel built-in driver.
  • Defining EXTRA_FIRMWARE_DIR and EXTRA_FIRMWARE Linux kernel configurations. Refer to drivers/base/firmware_loader/Kconfig for associated warning regarding the GPL licensing (alternative recommended by STMicroelectronics).
• The firmware name can be provided in the device tree using the property firmware-name (otherwise it must be named rproc-%s-fw, where %s corresponds to the name of the remoteproc node in the device tree). For example, for rproc-m4-fw, the remoteproc device tree must be defined as follows:

 m4 {
 	compatible = "st,stm32mp1-rproc";
 	[...]
 	status = "okay";
 };

• The "st,auto-boot" property has to be defined in the remoteproc node device tree:

 m4_rproc: m4@0  {
 	compatible = "st,stm32mp1-rproc";
 	[...]
 	st,auto-boot = <1>;
 	firmware-name = "my_firmware.elf";
 	status = "okay";
 };

4.1.3. Remote processor 'attach' boot

The Cortex-M processor can be 'early' started:

• By the Cortex-A second stage bootloader (e.g. U-Boot). This mode allows starting the Cortex-M processor firmware before the Linux one. For instance, it can be used to execute first actions for projects that have hard constraints on boot time.
• In M33-TD flavor for the STM32MP25x lines .

In such cases, the remoteproc framework can attach itself to the firmware by parsing the resource table, based on the information added by the bootloader in the backup registers (Cortex-M state and resource table address).

4.1.3.1. Automatic attach on Linux boot

On Linux boot, the remoteproc framework can automatically attach itself to the Cortex-M processor firmware if the st,auto-boot property is defined in the remoteproc node device tree:

 m4_rproc: m4@0  {
 	compatible = "st,stm32mp1-rproc";
 	[...]
 	st,auto-boot = <1>;
 	status = "okay";
 };

This mode is recommended to start the resource manager framework during Linux boot. It prevents a system resource, used by the Cortex-M only, from being disabled by Linux at the end of its boot phase.

4.1.3.2. Manual attach by Linux application

After Linux boot, the remoteproc framework can attach to the Cortex-M processor firmware using the following command:

 echo start > /sys/class/remoteproc/remoteprocX/state

Refer to How to start the Cortex-M coprocessor from U-Boot for details on this mode.

4.2. Remote processor stop

It is possible to stop the remote processor firmware through the sysFS interface. On stop request, the stm32_rproc driver:

• Informs the remote firmware relying on the "shutdown" channel of the IPCC mailbox. This mechanism allows the remote processor firmware to shut down properly.
• Resets the Cortex-M processor, on "shutdown" message acknowledgement or after a timeout of 500 ms.

Information

The use of the IPCC "shutdown" channel is optional. If the mailbox channel is not declared in the device tree, the remote processor is immediately reset, without informing the firmware of the remote processor.

To stop the firmware, use the following command:

 echo stop > /sys/class/remoteproc/remoteprocX/state

Information

Note that in STM32MPU Embedded Software distribution, a demonstration firmware is loaded by default. The load service can be stopped with the command: For STM32MP15x lines : systemctl stop st-m4firmware-load.service For STM32MP25x lines : systemctl stop st-m33firmware-load.service The load service can be disabled with the command: For STM32MP15x lines : systemctl disable st-m4firmware-load.service For STM32MP25x lines : systemctl disable st-m33firmware-load.service

4.3. Remote processor recovery on crash (not applicable for M33-TD flavor )

The remoteproc framework implements a recovery mechanism that allows it to automatically stop and reboot the remote processor, as well as the associated resources such as the RPMsg.

This event can be triggered by the Cortex-M early interrupt watchdogs by defining the associated interrupt in the device tree.

• STM32MP15x lines : The early interrupt of the WWDG watchdog can be defined:

 &m4_rproc {
 	interrupt-parent = <&exti>;
 	interrupts = <68 1>;
 };

• STM32MP2 series: The early interrupt of the following watchdogs can be defined:
  • IWDG3 or IWDG4 watchdog for the Cortex-M33
  • WWDG1 watchdog for the Cortex-M33
  • IWDG5 watchdog for the Cortex-M0+
  • WWDG2 watchdog for the Cortex-M0+

For compatibility with STM32MPU Embedded Software distribution, it is recommended to use the IWDG4 instance (or WWDG1 for strong real-time requirement).

 &m33_rproc {
 	interrupt-parent = <&intc>;
 	interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
 };

Information

The crash of the remote processor can be simulated for debug using the following command (replace <X> by the instance of the remote processor): echo 1 > /sys/kernel/debug/remoteproc/remoteprocX/crash

Information

The Cortex-M firmware is in charge of initializing and kicking the watchdog.

5. How to trace and debug the framework

5.1. How to monitor

The remoteproc firmware state can be monitored using the following command:

 cat /sys/class/remoteproc/remoteprocX/state

5.2. How to trace

Linux kernel remoteproc framework and driver debug traces can be added to the kernel logs using the dynamic debug mechanism:

 echo -n 'file stm32_rproc.c +p' > /sys/kernel/debug/dynamic_debug/control
 echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control

Cortex^®-M debug traces:

• On STM32MP15x lines :

A log buffer can be defined in the remoteproc firmware and declared in the resource table. If this feature is activated in the remote firmware, log traces from the Cortex-M can be dumped from the trace buffer using:

 cat /sys/kernel/debug/remoteproc/remoteprocX/trace0

• On STM32MP2 series:

The log buffer is not used; log traces are output on a dedicated UART.

5.3. Troubleshooting

Some common issues related to the management of a remote processor are listed in Cortex-M_remote_processor_management_troubleshooting_grid.

Linux^® is a registered trademark of Linus Torvalds.

Cortex^®

Inter-processor communication

Trusted execution environment

Open portable trusted execution environment

Executable and linkable file

Application programming interface

Device tree

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Microcontroller unit (MCUs have internal flash memory and are intended to operate with a minimum amount of external support ICs. They commonly are a self-contained, system-on-chip (SoC) designs.)

System control and management interface

Doubledata rate (memory domain)

Trusted firmware for Arm^® Cortex^®-M (see TF-M) - NEW

Arm^® Cortex^®-A35 trusted domain

Arm^® Cortex^®-M33 trusted domain

System file system (see https://en.wikipedia.org/wiki/Sysfs for more details)

Initial ramdisk (https://en.wikipedia.org/wiki/Initial_ramdisk)

Das U-Boot -- the universal boot loader (see U-Boot)

Inter-processor communication controller

Generic interrupt controller

Serial peripheral interface

Universal asynchronous receiver/transmitter

Linux^® is a registered trademark of Linus Torvalds.

Cortex^®

Inter-processor communication

Trusted execution environment

Open portable trusted execution environment

Executable and linkable file

Application programming interface

Device tree

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Microcontroller unit (MCUs have internal flash memory and are intended to operate with a minimum amount of external support ICs. They commonly are a self-contained, system-on-chip (SoC) designs.)

System control and management interface

Doubledata rate (memory domain)

Trusted firmware for Arm^® Cortex^®-M (see TF-M) - NEW

Arm^® Cortex^®-A35 trusted domain

Arm^® Cortex^®-M33 trusted domain

System file system (see https://en.wikipedia.org/wiki/Sysfs for more details)

Initial ramdisk (https://en.wikipedia.org/wiki/Initial_ramdisk)

Das U-Boot -- the universal boot loader (see U-Boot)

Inter-processor communication controller

Generic interrupt controller

Serial peripheral interface

Universal asynchronous receiver/transmitter