Difference between revisions of "Linux remoteproc framework overview"

[unchecked revision] [quality revision]
imported>Frq07267
m (Device tree configuration)


Template:ArticleMainWriter Template:ArticleApprovedVersion


SUMMARY
This article gives information about the Linux® remoteproc framework.

1 Framework purpose[edit]

The remote processor (RPROC) framework allows the different platforms/architectures to control (power on, load firmware, power off) remote processors while abstracting the hardware differences. In addition it offers services to monitor and debug the remote coprocessor.

2 System overview[edit]

Remoteproc overview.png

2.1 Component description[edit]

remoteproc: this is the remote processor framework generic part. Its role is to:

  • Load the ELF firmware in the remote processor memory.
  • Parse the firmware resource table to set associated resources (such as IPC communication, memory carevout and traces).
  • Control the remote processor execution (start, stop...).
  • Provide a service to monitor and debug the remote firmware.

stm32_rproc: this is the remote processor platform driver. Its role is to:

  • Register the vendor specific functions (callback) to the RPROC framework.
  • Handle the platform resources associated to the remote processor (such as registers, watchdogs, reset, clock and memories).
  • Forward notifications (kicks) to the remote processor through the mailbox framework.

2.2 API description[edit]

The API usage and remote processor binary firmware structure (resource table, ...) are described in the Linux kernel remoteproc documentation [1].

3 Configuration[edit]

3.1 Kernel configuration[edit]

Activate the remoteproc driver and framework in the kernel configuration using the Linux Menuconfig tool: Menuconfig or how to configure kernel.

Device drivers  --->
   Remoteproc drivers  --->
     <*> Support for Remote Processor subsystem
     <*> STM32 remoteproc support

3.2 Device tree configuration[edit]

The remote processor device node must be declared and enabled in the Linux kernel device tree. Here is an extract of the STM32MP1 evaluation board device tree:

 /* Memory region declaration, containing vring and rpmsg buffers */
 reserved-memory {
	#address-cells = <1>;
	#size-cells = <1>;
	ranges;
	/* RETRAM memory region reserved for firmware code and data */
        retram: retram@0x38000000 {
		compatible = "shared-dma-pool";
		reg = <0x38000000 0x10000>;
		no-map;
	};
	/* MCUSRAM memory region reserved for firmware code and data */
	mcusram: mcusram@0x10000000 {
		compatible = "shared-dma-pool";
		reg = <0x10000000 0x40000>;
		no-map;
	};
	/* MCUSRAM aliased memory region reserved for firmware code and data */
	mcusram2: mcusram2@0x30000000 {
		compatible = "shared-dma-pool";
		reg = <0x30000000 0x40000>;
		no-map;
	};

	/* memory region reserved for virtio ring buffer descriptor 0*/
	vdev0vring0: vdev0vring0@10040000 {
		compatible = "shared-dma-pool";
		reg = <0x10040000 0x2000>;
		no-map;
	};
	/* memory region reserved for virtio ring buffer descriptor 1*/
	vdev0vring1: vdev0vring1@10042000 {
		compatible = "shared-dma-pool";
		reg = <0x10042000 0x2000>;
		no-map;
	};
	/* memory region reserved for virtio buffers associated to the vrings*/
	vdev0buffer: vdev0buffer@10044000 {
		compatible = "shared-dma-pool";
		reg = <0x10044000 0x4000>;
		no-map;
	};
 };

  /* stm32 M4 remoteproc device */
  m4_rproc: m4@0 {
	compatible = "st,stm32mp1-rproc";
	ranges = <0x00000000 0x38000000 0x10000>,
		 <0x30000000 0x30000000 0x60000>,
		 <0x10000000 0x10000000 0x60000>;
	memory-region = <&retram>, <&mcusram>, <&mcusram2>, <&vdev0vring0>,
			<&vdev0vring1>, <&vdev0buffer>;
	mboxes = <&ipcc 0>, <&ipcc 1>, <&ipcc 2>;
	resets = <&rcc_rst MCU_R>;
	reset-names = "mcu_rst";
	st,hold_boot = <&rcc 0x10C 0x1>;
	st,smc_boot;
	mboxes = <&ipcc 0>, <&ipcc 1>, <&ipcc 2>;
	status = "okay";
 };
Info white.png Information
'memory-region' properties define the RETRAM and MCUSRAM base addresses and sizes in RETRAM and MCUSRAM, from the Arm® Cortex®-A point of view. The firmware memory mapping must be set according to these values in the linker script.

For additional details, please refer to STM32MP15 Memory mapping.

4 How to use the framework[edit]

There are three possibilities to load and start the remote processor firmware:

  • Start the firmware through the SysFS interface.
  • Automatically start the firmware on remoteproc driver probing (not recommended by STMicroelectronics).
  • Early boot the firmware during boot time (before Linux boot).

4.1 Remote processor boot through sysfs[edit]

  • The firmware components are stored in the file system, by default in the /lib/firmware/ folder. Optionally another location can be set. In this case the remoteproc framework parses this new path in priority.

Below the command for adding a new path for firmware parsing:

Board $> echo -n <firmware_path> > /sys/module/firmware_class/parameters/path
Warning white.png Warning
This path is common for all firmwares loaded by Linux (Bluetooth, Wifi...)
  • If the firmware elf filename differs from the default one (rproc-%s-fw), set the name with the following command: ( replace X with remoteproc instance number: 0 by default)
Board $> echo -n <firmware_name.elf> > /sys/class/remoteproc/remoteprocX/firmware
  • To start the firmware, use the following command:
Board $> echo start >/sys/class/remoteproc/remoteprocX/state
  • To stop the firmware, use the following command:
Board $> echo stop >/sys/class/remoteproc/remoteprocX/state
Info white.png Information
Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.

4.2 Remote processor 'auto' boot (not recommended by STMicroelectronics)[edit]

The remote processor can be automatically booted during platform boot. To do this, the following conditions must be fulfilled:

  • The firmware must be present in /lib/firmware before the remoteproc driver is probed.
  • The filesystem on Linux (Cortex-A) must be available before the remoteproc driver is probed. However, in normal conditions, the remoteproc driver is probed before the filesystem is mounted, and the firmware is consequently not available during the Linux driver probing phase. Possible solutions could be:
  • to use an initramfs[2]
  • or to put the remoteproc driver in the module.
  • The firmware must be named rproc-%s-fw, where %s corresponds to the name of the remoteproc node in the device tree. For example, for rproc-m4-fw.elf, the remoteproc device tree must be defined as follows:
m4 {
	compatible = "st,stm32mp1-rproc";
[...]
	status = "okay";
};
  • The "auto_boot" property has to be defined in the remoteproc node device tree:
m4 {
	compatible = "st,stm32mp1-rproc";
[...]
        auto_boot;
        status = "okay";
};

4.3 Remote processor 'early' boot[edit]

The coprocessor can be started by the second stage bootloader (eg U-Boot). This mode allows to start the coprocessor firmware before the Linux one. For instance, it can be used to execute first actions for projects that have hard constraints on boot time. On Linux boot, the remoteproc framework attaches itself to the firmware by parsing the resource table, based on the information added by the bootloader in the Linux device tree (resource table address and size). Refer to How to start the coprocessor from the bootloader for details on this mode.

5 How to trace and debug the framework[edit]

5.1 How to monitor[edit]

  • The remoteproc firmware state can be monitored using following command:
Board $> cat /sys/class/remoteproc/remoteprocX/state

5.2 How to trace[edit]

  • remoteproc framework and driver debug traces can be added in the kernel logs thanks to the dynamic debug mechanism:
Board $> echo -n 'file stm32_rproc.c +p' > /sys/kernel/debug/dynamic_debug/control
Board $> echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control
  • A log buffer can de defined in the remoteproc firmware and declared in the resource table. If the feature is activated on the remote firmware, log traces can be dumped from the trace buffer using the following command:
Board $>cat /sys/kernel/debug/remoteproc/remoteprocX/trace0

6 References[edit]

<noinclude>

{{ArticleBasedOnModel | [[Framework overview article model]]}}
{{ArticleMainWriter | ArnaudP}}
{{ArticleApprovedVersion | ArnaudP | FabienD (Passed,05Nov'18), GeraldB(Not Done), LoicP(Not Done), Jean-ChristopheT(Passed, 16Nov'18)|  14Mar'18 | AnneJ- 2Dec'18 - 9834 | 06Dec'18}} 

[[Category:Coprocessor management Linux]]</noinclude>


'''SUMMARY'''<br>

This article gives information about the Linux<sup>&reg;</sup> remoteproc framework.

==Framework purpose==
The remote processor (RPROC) framework allows the different platforms/architectures to control (power on, load firmware, power off) remote processors while abstracting the hardware differences. In addition it offers services to monitor and debug the remote coprocessor.

==System overview==
[[File:Remoteproc_overview.png|link=]]

===Component description===
'''remoteproc''': this is the remote processor framework generic part. Its role is to:
*Load the ELF firmware in the remote processor memory.
*Parse the firmware resource table to set associated resources (such as IPC communication, memory carevout and traces). 
*Control the remote processor execution (start, stop...).
*Provide a service to monitor and debug the remote firmware.

'''stm32_rproc''': this is the remote processor platform driver. Its role is to:
*Register the vendor specific functions (callback) to the RPROC framework.
*Handle the platform resources associated to the remote processor (such as registers, watchdogs, reset, clock and memories).
*Forward notifications (kicks) to the remote processor through the mailbox framework.

===API description===
The API usage and remote processor binary firmware structure (resource table, ...) are described in the Linux kernel remoteproc documentation <ref>[https://www.kernel.org/doc/Documentation/remoteproc.txt Linux kernel remoteproc documentation] </ref>.

==Configuration==
===Kernel configuration===
Activate the remoteproc driver and framework in the kernel configuration using the Linux Menuconfig tool: [[Menuconfig or how to configure kernel]].
 Device drivers  --->
    Remoteproc drivers  ---><*> Support for Remote Processor subsystem<*> STM32 remoteproc support

===Device tree configuration===
The remote processor device node must be declared and enabled in the Linux kernel [[Device tree|device tree]]. Here is an extract of the STM32MP1 evaluation board device tree:<pre>

 /* Memory region declaration, containing vring and rpmsg buffers */
 reserved-memory {
	#address-cells = <1>;
	#size-cells = <1>;
	ranges;
	/* RETRAM memory region reserved for firmware code and data */
        retram: retram@0x38000000 {
		compatible = "shared-dma-pool";
		reg = <0x38000000 0x10000>;
		no-map;
	};
	/* MCUSRAM memory region reserved for firmware code and data */
	mcusram: mcusram@0x10000000 {
		compatible = "shared-dma-pool";
		reg = <0x10000000 0x40000>;
		no-map;
	};
	/* MCUSRAM aliased memory region reserved for firmware code and data */
	mcusram2: mcusram2@0x30000000 {
		compatible = "shared-dma-pool";
		reg = <0x30000000 0x40000>;
		no-map;
	};

	/* memory region reserved for virtio ring buffer descriptor 0*/
	vdev0vring0: vdev0vring0@10040000 {
		compatible = "shared-dma-pool";
		reg = <0x10040000 0x2000>;
		no-map;
	};
	/* memory region reserved for virtio ring buffer descriptor 1*/
	vdev0vring1: vdev0vring1@10042000 {
		compatible = "shared-dma-pool";
		reg = <0x10042000 0x2000>;
		no-map;
	};
	/* memory region reserved for virtio buffers associated to the vrings*/
	vdev0buffer: vdev0buffer@10044000 {
		compatible = "shared-dma-pool";
		reg = <0x10044000 0x4000>;
		no-map;
	};
 };

  /* stm32 M4 remoteproc device */
  m4_rproc: m4@0 {
	compatible = "st,stm32mp1-rproc";
	ranges = <0x00000000 0x38000000 0x10000>,<0x30000000 0x30000000 0x60000>,<0x10000000 0x10000000 0x60000>;
	memory-region = <&retram>, <&mcusram>, <&mcusram2>, <&vdev0vring0>,<&vdev0vring1>, <&vdev0buffer>;
	mboxes = <&ipcc 0>, <&ipcc 1>, <&ipcc 2>;
	resets = <&rcc_rst MCU_R>;
	reset-names = "mcu_rst";
	st,hold_boot = <&rcc 0x10C 0x1>;
	st,smc_boot;
	mboxes = <&ipcc 0>, <&ipcc 1>, <&ipcc 2>;
	status = "okay";
 };</pre>


{{Info| 'memory-region' properties define the RETRAM and MCUSRAM base addresses and sizes in RETRAM and MCUSRAM, from the Arm<sup>&reg;</sup> Cortex<sup>&reg;</sup>-A point of view. The firmware memory mapping must be set according to these values in the linker script.
For additional details, please refer to [[STM32MP15_RAM_mapping#Memory_mapping | STM32MP15 Memory mapping]].}}

==How to use the framework==
There are three possibilities to load and start the remote processor firmware:
:* Start the firmware through the SysFS interface.
:* Automatically start the firmware on remoteproc driver probing (not recommended by STMicroelectronics).
:* Early boot the firmware during boot time (before Linux boot).  

=== Remote processor boot through sysfs===
* The firmware components are stored in the file system, by default in the '''/lib/firmware/''' folder. Optionally another location can be set. In this case the remoteproc framework parses this new path in priority.
Below the command for adding a new path for firmware parsing:
 Board $> echo -n {{RedHighlight|<firmware_path>}} > /sys/module/firmware_class/parameters/path
{{Warning | This path is common for all firmwares loaded by Linux (Bluetooth, Wifi...)}}

* If the firmware elf filename differs from the default one (rproc-%s-fw), set the name with the following command: ( replace {{RedHighlight|X}} with remoteproc instance number: 0 by default)
 Board $> echo -n {{RedHighlight|<firmware_name.elf>}} > /sys/class/remoteproc/remoteproc{{RedHighlight|X}}/firmware
* To start the firmware, use the following command:
 Board $> echo start >/sys/class/remoteproc/remoteproc{{RedHighlight|X}}/state
* To stop the firmware, use the following command:
 Board $> echo stop >/sys/class/remoteproc/remoteproc{{RedHighlight|X}}/state

{{Info| Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.}}

=== Remote processor 'auto' boot (not recommended by STMicroelectronics)===
The remote processor can be automatically booted during platform boot. To do this, the following conditions must be fulfilled:
*The firmware must be present in '''/lib/firmware''' before the remoteproc driver is probed.
*The filesystem on Linux (Cortex-A) must be available before the remoteproc driver is probed. However, in normal conditions, the remoteproc driver is probed before the filesystem is mounted, and the firmware is consequently not available during the Linux driver probing phase. Possible solutions could be:
:* to use an initramfs<ref>[https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt ramfs-rootfs-initramfs Linux documentation]</ref> 

:* or to put the remoteproc driver in the module.
*The firmware must be named '''rproc-%s-fw''', where '''%s''' corresponds to the name of the remoteproc node in the device tree. For example, for '''rproc-m4-fw.elf''', the remoteproc device tree must be defined as follows:
 {{RedHighlight|m4}} {
 	compatible = "st,stm32mp1-rproc";
 [...]
 	status = "okay";
 };
*The "auto_boot" property has to be defined in the remoteproc node device tree:
 m4 {
 	compatible = "st,stm32mp1-rproc";
 [...]
         {{RedHighlight|auto_boot}};
         status = "okay";
 };

=== Remote processor 'early' boot ===
The coprocessor can be started by the second stage bootloader (eg U-Boot). This mode allows to start the coprocessor firmware before the Linux one. For instance, it can be used to execute first actions for projects that have hard constraints on boot time.
On Linux boot, the remoteproc framework attaches itself to the firmware by parsing the resource table, based on the information added by the bootloader in the Linux device tree (resource table address and size). 
Refer to [[How to start the coprocessor from the bootloader]] for details on this mode.

==How to trace and debug the framework==
=== How to monitor ===
* The remoteproc firmware state can be monitored using following command:
 Board $> cat /sys/class/remoteproc/remoteproc{{RedHighlight|X}}/state
=== How to trace ===
* remoteproc framework and driver debug traces can be added in the kernel logs thanks to the [[How_to_use_the_kernel_dynamic_debug|dynamic debug]] mechanism:
 Board $> echo -n 'file stm32_rproc.c +p' > /sys/kernel/debug/dynamic_debug/control
 Board $> echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control
* A log buffer can de defined in the remoteproc firmware and declared in the resource table. If the feature is activated on the  remote firmware, log traces can be dumped from the trace buffer using the following command:
 Board $>cat /sys/kernel/debug/remoteproc/remoteproc{{RedHighlight|X}}/trace0

==References==
<references />
Line 116: Line 116:
 
* The firmware components are stored in the file system, by default in the '''/lib/firmware/''' folder. Optionally another location can be set. In this case the remoteproc framework parses this new path in priority.
 
* The firmware components are stored in the file system, by default in the '''/lib/firmware/''' folder. Optionally another location can be set. In this case the remoteproc framework parses this new path in priority.
 
Below the command for adding a new path for firmware parsing:
 
Below the command for adding a new path for firmware parsing:
  Board $> echo -n {{Red|<firmware_path>}} > /sys/module/firmware_class/parameters/path
+
  Board $> echo -n {{Highlight|<firmware_path>}} > /sys/module/firmware_class/parameters/path
 
{{Warning | This path is common for all firmwares loaded by Linux (Bluetooth, Wifi...)}}
 
{{Warning | This path is common for all firmwares loaded by Linux (Bluetooth, Wifi...)}}
   
* If the firmware elf filename differs from the default one (rproc-%s-fw), set the name with the following command: ( replace {{Red|X}} with remoteproc instance number: 0 by default)
+
* If the firmware elf filename differs from the default one (rproc-%s-fw), set the name with the following command: ( replace {{Highlight|X}} with remoteproc instance number: 0 by default)
  Board $> echo -n {{Red|<firmware_name.elf>}} > /sys/class/remoteproc/remoteproc{{Red|X}}/firmware
+
  Board $> echo -n {{Highlight|<firmware_name.elf>}} > /sys/class/remoteproc/remoteproc{{Highlight|X}}/firmware
 
* To start the firmware, use the following command:
 
* To start the firmware, use the following command:
  Board $> echo start >/sys/class/remoteproc/remoteproc{{Red|X}}/state
+
  Board $> echo start >/sys/class/remoteproc/remoteproc{{Highlight|X}}/state
 
* To stop the firmware, use the following command:
 
* To stop the firmware, use the following command:
  Board $> echo stop >/sys/class/remoteproc/remoteproc{{Red|X}}/state
+
  Board $> echo stop >/sys/class/remoteproc/remoteproc{{Highlight|X}}/state
   
 
{{Info| Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.}}
 
{{Info| Based on the above commands, a userland service can be implemented to automatically load the firmware during the userland initialization phase.}}
Line 135: Line 135:
 
:* or to put the remoteproc driver in the module.
 
:* or to put the remoteproc driver in the module.
 
*The firmware must be named '''rproc-%s-fw''', where '''%s''' corresponds to the name of the remoteproc node in the device tree. For example, for '''rproc-m4-fw.elf''', the remoteproc device tree must be defined as follows:
 
*The firmware must be named '''rproc-%s-fw''', where '''%s''' corresponds to the name of the remoteproc node in the device tree. For example, for '''rproc-m4-fw.elf''', the remoteproc device tree must be defined as follows:
  {{Red|m4}} {
+
  {{Highlight|m4}} {
 
  compatible = "st,stm32mp1-rproc";
 
  compatible = "st,stm32mp1-rproc";
 
  [...]
 
  [...]
Line 144: Line 144:
 
  compatible = "st,stm32mp1-rproc";
 
  compatible = "st,stm32mp1-rproc";
 
  [...]
 
  [...]
         {{Red|auto_boot}};
+
         {{Highlight|auto_boot}};
 
         status = "okay";
 
         status = "okay";
 
  };
 
  };
Line 156: Line 156:
 
=== How to monitor ===
 
=== How to monitor ===
 
* The remoteproc firmware state can be monitored using following command:
 
* The remoteproc firmware state can be monitored using following command:
  Board $> cat /sys/class/remoteproc/remoteproc{{Red|X}}/state
+
  Board $> cat /sys/class/remoteproc/remoteproc{{Highlight|X}}/state
 
=== How to trace ===
 
=== How to trace ===
 
* remoteproc framework and driver debug traces can be added in the kernel logs thanks to the [[How_to_use_the_kernel_dynamic_debug|dynamic debug]] mechanism:
 
* remoteproc framework and driver debug traces can be added in the kernel logs thanks to the [[How_to_use_the_kernel_dynamic_debug|dynamic debug]] mechanism:
Line 162: Line 162:
 
  Board $> echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control
 
  Board $> echo -n 'file remoteproc*.c +p' > /sys/kernel/debug/dynamic_debug/control
 
* A log buffer can de defined in the remoteproc firmware and declared in the resource table. If the feature is activated on the  remote firmware, log traces can be dumped from the trace buffer using the following command:
 
* A log buffer can de defined in the remoteproc firmware and declared in the resource table. If the feature is activated on the  remote firmware, log traces can be dumped from the trace buffer using the following command:
  Board $>cat /sys/kernel/debug/remoteproc/remoteproc{{Red|X}}/trace0
+
  Board $>cat /sys/kernel/debug/remoteproc/remoteproc{{Highlight|X}}/trace0
   
 
==References==
 
==References==
   
 
<references />
 
<references />