Difference between revisions of "How to exchange large data buffers with the coprocessor - principle"

[quality revision] [quality revision]
 

1 Introduction[edit]

As explained in "Exchanging_buffers_with_the_coprocessor", the RPMsg protocol may be not efficient enough to directly exchange large buffers between the Cortexes. In this case implementing the indirect buffer exchange mode is recommended:

  • allocate contiguous buffers to store the data to exchange
  • use RPMsg to exchange references to these buffers with the remote processor.

This article gives an example mechanism that can be implemented to exchange indirect buffers between a main processor and a coprocessor.

2 Architecture overview[edit]

This architecture example relies on the rpmsg_sdb linux driver.

Copro-sw-ipc-big-data.png

  • On Cortex-A:
  • The rpmsg_sdb Linux driver implements the service to allocate and share buffers with the Cortex-M.
  • The Linux application requests and memory-maps (mmap) the buffers needed to access the associated memory.
  • On Cortex-M:
  • The application has to implement the "rpmsg-sdb-channel" RPMsg service to manage the buffer information.
  • A DMA can be used to transfer data to/from DDR.

Refer to How to exchange large data buffers with the coprocessor - example for an example.

3 rpmsg_sdb driver[edit]

The RPMsg shared data buffer driver example is in charge of:

  • Allocating large buffers in contiguous memory (DDR) and memory mapping them (mmap) for use by an application.
  • implementing the RPMsg service to share buffer information (address, size) with the coprocessor.
  • Sending events to a Linux application (relying on the eventfd interface) when buffers are available (on RPMsg message reception).
Info.png This driver is provided as example. It implements only the transfer from Cortex-M to cortex-A

3.1 Configuration[edit]

3.1.1 Kernel configuration[edit]

No kernel configuration is needed. The rpmsg_sdb Linux driver is proposed as module and can be installed using the associated Yocto recipe.

3.1.2 Device tree[edit]

No device tree declaration is needed. The rpmsg_sdb driver is registered as an RPMsg driver. it is probed when the remote processor creates the "rpmsg-sdb-channel" service.

3.1.3 Source code[edit]

The source code is available in the rpsmg-sdb-mod Yocto recipe.

3.2 How to use[edit]

3.2.1 User space interface[edit]

The rpmsg_sdb driver exposes a "/dev/rpmsg_sdb" sysfs that offers an interface to allocate and manage the shared buffers.

  • open/close: get/release file descriptor.
  int fd;
  fd= open('/dev/rpmsg_sdb');
  close(fd);

  • mmap: allocate and map memories
  void *buff0_id, *buff1_id;
 
  buff0_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);
  buff1_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);

  • RPMSG_SDB_IOCTL_SET_EFD ioctl: register event for a buffer
  typedef struct
  {
  	int bufferId, eventfd;
  } rpmsg_sdb_ioctl_set_efd;
  
  int efd[NB_BUF];
  rpmsg_sdb_ioctl_set_efd q_set_efd;

  for (i=0;i<NB_BUF;i++){
	/* Create the evenfd, and sent it to kernel driver, for notification of buffer full */
	efd[i] = eventfd(0, 0);
  }
  q_set_efd.bufferId = i;  /* i is the index of the buffer */
  q_set_efd.eventfd = efd[i];
  
  ioctl(fd, RPMSG_SDB_IOCTL_SET_EFD, buff0_id, &q_set_efd);

  • RPMSG_SDB_IOCTL_GET_DATA_SIZE ioctl : get the size of a buffer
  typedef struct
  {
  	int bufferId;
  	uint32_t size;
  } rpmsg_sdb_ioctl_get_data_size;
  
  rpmsg_sdb_ioctl_get_data_size q_get_data_size;

  ioctl(fd, RPMSG_SDB_IOCTL_GET_DATA_SIZE, buff0_id, eventfd);

p*manage event

  while (1) {
	ret = poll(fds, NB_BUF, TIMEOUT * 1000);
	if (ret < 0) {
		perror("poll()");
		break;
	} else if (ret) {
		printf("Data is available now.\n");
	} else if (ret == 0) {
		printf("No data within five seconds.\n");
		break;
	}
	for (j=0;j<NB_BUF;j++){
		if (fds[j].revents & POLLIN) {
			/* Event received for the buffer j: New data is available for buffer j */
		}
	}
}

3.2.2 RPMsg messaging[edit]

The RPMsg protocol is used for communication with the Cortex-M:

  • Information about the buffer allocated and mmaped is sent to the Cortex-M.
The message is structured in a string with following format: "BxAyyyyyyyyLzzzzzzzz"
  • x: buffer index (32 bits, decimal format, no leading zero)
  • yyyyyyyy: physical address of the buffer in DDR (32 bits, 8-digit hexadecimal format, leading zero)
  • zzzzzzzz: length of the buffer (32 bits, 8-digit hexadecimal format, leading zero).
  • Buffer filled event received from the Cortex-M:
When the Cortex-M4 has filled a buffer it can inform the Linux application by sending an RPMsg with following string format : "BxLzzzzzzzz"'.
  • x: buffer index (32 bits, decimal format, no leading zero)
  • zzzzzzzz: length of the buffer (32 bits, 8-digit hexadecimal format, leading zero).
On reception of this message the rpmsg_sdb driver sends an event to the application.



== Introduction ==
As explained in "[[Exchanging_buffers_with_the_coprocessor]]", the RPMsg  protocol may be not efficient enough to directly exchange large buffers between the Cortexes. In this case implementing the indirect buffer exchange mode is recommended:
* allocate contiguous buffers to store the data to exchange
* use RPMsg to exchange references to these buffers with the remote processor.
This article gives an example mechanism that can be implemented to exchange indirect buffers between a main processor and a coprocessor.

== Architecture overview ==
This architecture example relies on the rpmsg_sdb linux driver. 

[[File:copro-sw-ipc-big-data.png | |link=]]
* <u>On Cortex-A</u>: 
:* The rpmsg_sdb Linux driver implements the service to allocate and share buffers with the Cortex-M.
:* The Linux application requests and memory-maps (mmap) the buffers needed to access the associated memory.
* <u>On Cortex-M</u>: 
:*The application has to implement the "rpmsg-sdb-channel"  RPMsg service to manage the buffer information. 
:*A DMA can be used to transfer data to/from DDR.
Refer to [[How_to_exchange_large_data_buffers_with_the_coprocessor_-_example| How to exchange large data buffers with the coprocessor - example]] for an example.

== rpmsg_sdb driver ==
The RPMsg shared data buffer driver example is in charge of:
* Allocating large buffers in contiguous memory (DDR) and memory mapping them (mmap) for use by an application.
* implementing the RPMsg service to share buffer information (address, size)  with the coprocessor.
* Sending events to a Linux application (relying on the [http://man7.org/linux/man-pages/man2/eventfd.2.html eventfd] interface) when buffers are available (on RPMsg message reception).

{{Info| This driver is provided as example. It implements only the transfer from Cortex-M to cortex-A}}

=== Configuration ===
==== Kernel configuration ====
No kernel configuration is needed. The rpmsg_sdb Linux driver is proposed as module and can be installed using the associated [https://github.com/STMicroelectronics/meta-st-stm32mpu-app-logicanalyser/tree/thud/recipes-kernel/rpsmg-sdb-mod  Yocto recipe].

==== Device tree ====
No device tree declaration is needed. The rpmsg_sdb driver is registered as an RPMsg driver. it is probed when the remote processor creates the "rpmsg-sdb-channel" service.
==== Source code ====
The source code is available in the [https://github.com/STMicroelectronics/meta-st-stm32mpu-app-logicanalyser/tree/thud/recipes-kernel/rpsmg-sdb-mod  rpsmg-sdb-mod Yocto recipe].

=== How to use ===
==== User space interface ====
The rpmsg_sdb driver exposes a '''"/dev/rpmsg_sdb"''' sysfs that offers an interface to allocate and manage the shared buffers. 

* open/close: get/release file descriptor.<syntaxhighlight lang="c">

  int fd;
  fd= open('/dev/rpmsg_sdb');
  close(fd);</syntaxhighlight>


* mmap: allocate and map memories<syntaxhighlight lang="c">

  void *buff0_id, *buff1_id;

  buff0_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);
  buff1_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);</syntaxhighlight>


* RPMSG_SDB_IOCTL_SET_EFD ioctl: register event for a buffer<syntaxhighlight lang="c">

  typedef struct
  {
  	int bufferId, eventfd;
  } rpmsg_sdb_ioctl_set_efd;

  int efd[NB_BUF];
  rpmsg_sdb_ioctl_set_efd q_set_efd;

  for (i=0;i<NB_BUF;i++){
	/* Create the evenfd, and sent it to kernel driver, for notification of buffer full */
	efd[i] = eventfd(0, 0);
  }
  q_set_efd.bufferId = i;  /* i is the index of the buffer */
  q_set_efd.eventfd = efd[i];

  ioctl(fd, RPMSG_SDB_IOCTL_SET_EFD, buff0_id, &q_set_efd);</syntaxhighlight>

*RPMSG_SDB_IOCTL_GET_DATA_SIZE ioctl : get the size of a buffer<syntaxhighlight lang="c">

  typedef struct
  {
  	int bufferId;
  	uint32_t size;
  } rpmsg_sdb_ioctl_get_data_size;

  rpmsg_sdb_ioctl_get_data_size q_get_data_size;

  ioctl(fd, RPMSG_SDB_IOCTL_GET_DATA_SIZE, buff0_id, eventfd);</syntaxhighlight>

p*manage event<syntaxhighlight lang="c">

  while (1) {
	ret = poll(fds, NB_BUF, TIMEOUT * 1000);
	if (ret < 0) {
		perror("poll()");
		break;
	} else if (ret) {
		printf("Data is available now.\n");
	} else if (ret == 0) {
		printf("No data within five seconds.\n");
		break;
	}
	for (j=0;j<NB_BUF;j++){
		if (fds[j].revents & POLLIN) {
			/* Event received for the buffer j: New data is available for buffer j */
		}
	}
}</syntaxhighlight>

==== RPMsg messaging ====
The RPMsg protocol is used for communication with the Cortex-M:
* Information about the buffer allocated and mmaped is sent to the Cortex-M.
:The message is structured in a string with following format: '''"BxAyyyyyyyyLzzzzzzzz"'''
:*'''x''': buffer index (32 bits, decimal format, no leading zero)
:*'''yyyyyyyy''': physical address of the buffer in DDR (32 bits, 8-digit hexadecimal format, leading zero)
:*'''zzzzzzzz''': length of the buffer (32 bits, 8-digit hexadecimal format, leading zero).
* Buffer filled event received from the Cortex-M:
:When the Cortex-M4 has filled a buffer it can inform the Linux application by sending an RPMsg with following string format : '''"BxLzzzzzzzz"''''.
:*'''x''': buffer index (32 bits, decimal format, no leading zero)
:*'''zzzzzzzz''': length of the buffer (32 bits, 8-digit hexadecimal format, leading zero).
:On reception of this message the rpmsg_sdb driver sends an event to the application.
<noinclude>

{{PublicationRequestId | 14367| AnneJ| 2019-12-10}}[[Category:How to run use cases with expansions]]
[[Category:Coprocessor_management_Linux]]
[[Category:Coprocessor_management_STM32Cube]]</noinclude>
Line 121: Line 121:
 
<noinclude>
 
<noinclude>
 
{{PublicationRequestId | 14367| AnneJ| 2019-12-10}}
 
{{PublicationRequestId | 14367| AnneJ| 2019-12-10}}
  +
[[Category:How to run use cases with expansions]]
  +
[[Category:Coprocessor_management_Linux]]
  +
[[Category:Coprocessor_management_STM32Cube]]
 
</noinclude>
 
</noinclude>