How to exchange large data buffers with the coprocessor - principle

1. Introduction

As explain In "How_to_exchange_buffers_with_the_Cortex-M_co-processor", the rpmsg protocol can not be enough efficient to directly exchange large buffers between the Cortexes. In this case it is recommended to implement the indirect buffer exchange mode:

• Allocate contiguous buffers to store the data to exchange,
• use RPMsg to exchange reference to these buffers with the remote processor.

This article gives an example of mechanism that can be put in place to exchange indirect buffers between a main processor and a coprocessor.

2. Architecture overview

This architecture example is mainly relying on a the rpmsg_sdb linux driver.

• On Cortex-A:

• The rpmsg_sdb driver implements on Linux side the service to allocate and share buffers with the Cortex-M
• the Linux Application requests and mmaps the buffers to have be able to accesses to the associated memory.

• On Cortex-M:

• Application has to implement the "rpmsg-sdb-channel" rpmsg service to manage the buffer information.
• a DMA can be used to transfer data to/from DDR.

For example, refer to How_to_exchange_big_data_between_Cortex-M4_and_Cortex-A7 article.

3. rpmsg_sdb driver

The RPMsg shared data buffer driver example in in charge of:

• allocating large buffers in contiguous memory(DDR) and mmap them for an application usage.
• implementing the RPMsg service to share buffer information (address, size) with the coprocessor.
• send event to Linux Application (relying on eventfd interface}) when buffers are available (on RPMsg related message reception).

Information

This drivers is only provided as example, Only the transfer from Cortex-M to cortex-A is implemented in the rpmsg_sdb driver

3.1. Configuration

3.1.1. Kernel configuration

No kernel configuration is needed. The rpmsg_sdb Linux driver is proposed as module and can be installed thanks to the associated Yocto recipe.

3.1.2. Device tree

No device tree device tree declaration is needed. The rpmsg_sdb driver is registered as a rpmsg driver. it is probed when the remote processor creates the "rpmsg-sdb-channel" service

3.1.3. Source code

the source code is available in the rpsmg-sdb-mod Yocto recipe.

3.2. How to use

3.2.1. User space interface

The rpmsg_sdb driver exposes a "/dev/rpmsg_sdb" sysfs that offers an interface to allocate and manage the shared buffers.

• open/close: get/release file descriptor.

  int fd;
  fd= open('/dev/rpmsg_sdb');
  close(fd);

• mmap: allocate and map memories

  void *buff0_id, *buff1_id;
 
  buff0_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);
  buff1_id = mmap(NULL, size,  PROT_READ | PROT_WRITE, 	MAP_PRIVATE, fd, 0);

• ioctrl

• RPMSG_SDB_IOCTL_SET_EFD

register event for a buffer

  typedef struct
  {
  	int bufferId, eventfd;
  } rpmsg_sdb_ioctl_set_efd;
  
  int efd[NB_BUF];
  rpmsg_sdb_ioctl_set_efd q_set_efd;

  for (i=0;i<NB_BUF;i++){
	/* Create the evenfd, and sent it to kernel driver, for notification of buffer full */
	efd[i] = eventfd(0, 0);
  }
  q_set_efd.bufferId = i;  /* i is the index of the buffer */
  q_set_efd.eventfd = efd[i];
  
  ioctrl(fd, RPMSG_SDB_IOCTL_SET_EFD, buff0_id, &q_set_efd);

• RPMSG_SDB_IOCTL_GET_DATA_SIZE

get the size of a buffer

  typedef struct
  {
  	int bufferId;
  	uint32_t size;
  } rpmsg_sdb_ioctl_get_data_size;
  
  rpmsg_sdb_ioctl_get_data_size q_get_data_size;

  ioctrl(fd, RPMSG_SDB_IOCTL_GET_DATA_SIZE, buff0_id, eventfd);

• manage event

  while (1) {
	ret = poll(fds, NB_BUF, TIMEOUT * 1000);
	if (ret < 0) {
		perror("poll()");
		break;
	} else if (ret) {
		printf("Data is available now.\n");
	} else if (ret == 0) {
		printf("No data within five seconds.\n");
		break;
	}
	for (j=0;j<NB_BUF;j++){
		if (fds[j].revents & POLLIN) {
			/* Event received for the buffer j: New data is available for buffer j */
		}
	}
}

3.2.2. RPMsg messaging

The RPMsg protocol is used for the communication with the Cortex-M:

• Information on buffer allocated and mmaped is sent to the Cortex-M.

The message is structured in a string with following format: "BxAyyyyyyyyLzzzzzzzz"

• x: buffer index,
• yyyyyyyy: address of the buffer in DDR ( 32 bits format),
• zzzzzzzz: length of the buffer (32 bits format)

• Buffer filled event received from the Cortex-M:

When the Cortex-M4 has filled a buffer it can inform the Linux Application by sending a RPMsg with following string Format : "BxLzzzzzzzz"'.

• x: buffer index,
• zzzzzzzz: length of the buffer (32 bits format)

On this message reception the rpmsg_sdb driver sends an event to application .

Remote Processor Messaging

Cortex^®

Linux^® is a registered trademark of Linus Torvalds.

Direct Memory Access

Doubledata rate (memory domain)

System File System (See https://en.wikipedia.org/wiki/Sysfs for more details)