How to cross-compile with the Developer Package

Applicable for STM32MP13x lines, STM32MP15x lines


1 Article purpose[edit]

This article provides simple examples for the Developer Package of the OpenSTLinux distribution, that illustrate cross-compilation with the SDK:

  • modification of software elements delivered as source code (for example the Linux kernel)
  • addition of software (for example the Linux kernel module or user-space applications)

These examples also show how to deploy the results of the cross-compilation on the target, through a network connection to the host machine.

Info white.png Information
There are many ways to achieve the same result; this article aims to provide at least one solution per example. You are free to explore other methods that are better adapted to your development constraints.

2 Prerequisites[edit]

The prerequisites from the Cross-compile with OpenSTLinux SDK article must be executed, and the cross-compilation and deployment of any piece of software, as explained in that article, is known.

The board and the host machine are connected through an Ethernet link, and a remote terminal program is started on the host machine: see How to get Terminal.

The target is started, and its IP address (<board ip address>) is known.

Info white.png Information
If you encounter a problem with any of the commands in this article, remember that the README.HOW_TO.txt helper files, from the Linux kernel, Optee-OS, TF-A and U-Boot installation directories, are the build references.
Info white.png Information
Regarding the Linux kernel examples, it is considered that the Linux kernel has been setup, configured and built a first time in a dedicated build directory (<Linux kernel build directory> later in this page) different from the source code directory (<Linux kernel source directory> later in this document).

3 Modifying the Linux kernel configuration[edit]

3.1 Preamble[edit]

Warning white.png Warning
Please read carefully and pay attention to the following point before modifying the Linux kernel configuration

The Linux kernel configuration option that you want to modify might be used by external out-of-tree Linux kernel modules (for example the GPU kernel driver), and these should then be recompiled. These modules are, by definition, outside the kernel tree structure, and are not delivered in the Developer Package source code; it is not possible to recompile them with the Developer Package. Consequently, if the Linux kernel is reconfigured and recompiled with this option then deployed on the board, the external out-of-tree Linux kernel modules might no longer be loaded.

There are two possible situations:

  • This is not a problem for the use cases on which you are currently working. In this case you can use the Developer Package to modify and recompile the Linux kernel.
  • This is a problem for the use cases on which you are currently working. In this case you need to switch on the STM32MP1 Distribution Package, and after having modified the Linux kernel configuration, use it to rebuild the whole image (that is, not only the Linux kernel but also the external out-of-tree Linux kernel modules).

Example:

  • Let's assume that the FUNCTION_TRACER and FUNCTION_GRAPH_TRACER options are activated to install the ftrace Linux kernel feature
  • This feature is used to add tracers in the whole kernel, including the external out-of-tree Linux kernel modules
  1. The Developer Package is used to reconfigure and recompile the Linux kernel, and to deploy it on the board
    1. The external out-of-tree Linux kernel modules are not recompiled. This is the case for the GPU kernel driver
    2. Consequently, the Linux kernel fails to load the GPU kernel driver module. However, even if the display no longer works, the Linux kernel boot succeeds, and the setup is sufficient, for example, to debug use cases involving an Ethernet or USB connection
  2. The Distribution Package is used to reconfigure the Linux kernel, and to rebuild and deploy the whole image on the board
    1. The external out-of-tree Linux kernel modules are recompiled, including the GPU kernel driver
    2. Consequently, the Linux kernel succeeds in loading the GPU kernel driver module. The display is available.

3.2 Simple example[edit]

This simple example modifies the value defined for the contiguous memory area (CMA) size.

  • Get the current value of the CMA size (128 Mbytes here) through the analysis of the target boot log
 dmesg | grep -i cma
STM32MP135F-DK More info green.png
[    0.000000] cma: Reserved 128 MiB at 0x...
STM32MP157F-EV1 More info green.png
[    0.000000] cma: Reserved 128 MiB at 0xe8000000
STM32MP157F-DK2 More info green.png
[    0.000000] cma: Reserved 128 MiB at 0xd2000000
  • Go to the Linux kernel build directory
 cd <Linux kernel build directory>
  • Navigate to "Device Drivers - Generic Driver Options"
    • select "Size in Megabytes"
    • modify its value to 256
    • exit and save the new configuration
  • Check that the configuration file (.config) has been modified
 grep -i CONFIG_CMA_SIZE_MBYTES .config
CONFIG_CMA_SIZE_MBYTES=256
  • Get the new value of the CMA size (256 Mbytes) through the analysis of the target boot log
 dmesg | grep -i cma
STM32MP135F-DK More info green.png
[    0.000000] cma: Reserved 256 MiB at 0x...
STM32MP157F-EV1 More info green.png
[    0.000000] cma: Reserved 256 MiB at 0xd8000000
STM32MP157F-DK2 More info green.png
[    0.000000] cma: Reserved 256 MiB at 0xc2000000

4 Modifying the Linux kernel device tree[edit]

This simple example modifies the default status of a user LED.

  • With the board started; check that the user green LED (LD3 for STM32MP157F-EV1 More info green.png, LD5 for STM32MP157F-DK2 More info green.png) is disabled
  • Go to the Linux kernel source directory
 cd <Linux kernel source directory>
  • Edit the device tree source file to add the lines highlighted below
STM32MP157F-EV1 More info green.png arch/arm/boot/dts/stm32mp157f-ed1.dts
	led {
		compatible = "gpio-leds";
		led-blue {
			label = "heartbeat";
			gpios = <&gpiod 9 GPIO_ACTIVE_HIGH>;
			linux,default-trigger = "heartbeat";
			default-state = "off";
		};
 		led-green {
 			label = "stm32mp:green:user";
 			gpios = <&gpioa 14 GPIO_ACTIVE_LOW>;
 			default-state = "on";
  		};
	};
STM32MP157F-DK2 More info green.png arch/arm/boot/dts/stm32mp15xx-dkx.dtsi
	led {
		compatible = "gpio-leds";
		led-blue {
			label = "heartbeat";
			gpios = <&gpiod 11 GPIO_ACTIVE_HIGH>;
			linux,default-trigger = "heartbeat";
			default-state = "off";
		};
 		led-green {
 			label = "stm32mp:green:user";
 			gpios = <&gpioa 14 GPIO_ACTIVE_LOW>;
 			default-state = "on";
  		};
	};


  • Go to the Linux kernel build directory
 cd <Linux kernel build directory>
  • Generate the device tree blobs (*.dtb)
 make dtbs
 cp arch/arm/boot/dts/stm32mp157*.dtb install_artifact/boot/
  • Update the device tree blobs on the board
 scp install_artifact/boot/stm32mp157*.dtb root@<board ip address>:/boot/
Info white.png Information
If the /boot mounting point doesn't exist yet, please see how to create a mounting point
  • Reboot the board
 reboot
  • Check that the user green LED (LD3 for STM32MP157F-EV1 More info green.png, LD5 for STM32MP157F-DK2 More info green.png) is enabled (green)

5 Modifying a built-in Linux kernel device driver[edit]

This simple example adds unconditional log information when the display driver is probed.

  • Check that there's no log information when the display driver is probed
 dmesg | grep -i stm_drm_platform_probe

  • Go to the Linux kernel source directory
 cd <Linux kernel source directory>
  • Edit the ./drivers/gpu/drm/stm/drv.c source file
  • Add a log information in the stm_drm_platform_probe function
static int stm_drm_platform_probe(struct platform_device *pdev)
{
	struct device *dev = &pdev->dev;
	struct drm_device *ddev;
	int ret;
	[...]

	DRM_INFO("Simple example - %s\n", __func__);

	return 0;
	[...]
}
  • Go to the Linux kernel build directory
 cd <Linux kernel build directory>
  • Cross-compile the Linux kernel (please check the load address in the README.HOW_TO.txt helper file)
 make uImage LOADADDR=0xC2000040
 cp arch/arm/boot/uImage install_artifact/boot/
  • Update the Linux kernel image on board
 scp install_artifact/boot/uImage root@<board ip address>:/boot/
Info white.png Information
If the /boot mounting point doesn't exist yet, please see how to create a mounting point
  • Reboot the board
 reboot
  • Check that there is now log information when the display driver is probed
 dmesg | grep -i stm_drm_platform_probe
[    2.995125] [drm] Simple example - stm_drm_platform_probe

6 Modifying/adding an external Linux kernel module[edit]

Most device drivers (modules) in the Linux kernel can be compiled either into the kernel itself (built-in/internal module) or as Loadable Kernel Modules (LKM/external module) that need to be placed in the root file system under the /lib/modules directory. An external module can be in-tree (in the kernel tree structure), or out-of-tree (outside the kernel tree structure).

6.1 Modifying an external in-tree Linux kernel module[edit]

This simple example adds an unconditional log information when the virtual video test driver (vivid) kernel module is probed or removed.

  • Go to the Linux kernel source directory
 cd <Linux kernel source directory>
  • Edit the ./drivers/media/test-drivers/vivid/vivid-core.c source file
  • Add log information in the vivid_probe and vivid_remove functions
static int vivid_probe(struct platform_device *pdev)
{
	const struct font_desc *font = find_font("VGA8x16");
	int ret = 0, i;
	[...]

	/* n_devs will reflect the actual number of allocated devices */
	n_devs = i;

	pr_info("Simple example - %s\n", __func__);

	return ret;
}
static int vivid_remove(struct platform_device *pdev)
{
	struct vivid_dev *dev;
	unsigned int i, j;
	[...]

	pr_info("Simple example - %s\n", __func__);

	return 0;
}
  • Go to the Linux kernel build directory
 cd <Linux kernel build directory>

  • Cross-compile the Linux kernel modules
 make modules
 make INSTALL_MOD_PATH="./install_artifact" modules_install
  • Remove the link on install_artifact/lib/modules/<kernel version>/
 rm install_artifact/lib/modules/<kernel version>/build
 rm install_artifact/lib/modules/<kernel version>/source
  • Optionally, strip kernel modules (to reduce the size of each kernel modules)
 find . -name "*.ko" | xargs $STRIP --strip-debug --remove-section=.comment --remove-section=.note --preserve-dates
  • Update the vivid kernel module on the board (please check the kernel version <kernel version>)
 scp install_artifact/lib/modules/"<kernel version>"/kernel/drivers/media/test-drivers/vivid/vivid.ko root@<board ip address>:/lib/modules/<kernel version>/kernel/drivers/media/test-drivers/vivid/

OR

 scp -r install_artifact/lib/modules/* root@<board ip address>:/lib/modules/
  • Update dependency descriptions for loadable kernel modules, and synchronize the data on disk with memory
 /sbin/depmod -a
 sync
  • Insert the vivid kernel module into the Linux kernel
 modprobe vivid                   
[...]
[ 3412.784638] Simple example - vivid_probe
  • Remove the vivid kernel module from the Linux kernel
 rmmod vivid
[...]
[ 3423.708517] Simple example - vivid_remove

6.2 Adding an external out-of-tree Linux kernel module[edit]

This simple example adds a "Hello World" external out-of-tree Linux kernel module to the Linux kernel.

  • Prerequisite: the Linux source code is installed, and the Linux kernel has been cross-compiled
  • Go to the working directory that contains all the source code (that is, the directory that contains the Linux kernel, Optee-OS, TF-A and U-Boot source code directories)
 cd <tag>/sources/arm-<distro>-linux-gnueabi
  • Export to KERNEL_SRC_PATH the path to the Linux kernel build directory that contains both the Linux kernel source code and the configuration file (.config)
 export KERNEL_SRC_PATH=$PWD/<Linux kernel build directory>/
  • Create a directory for this kernel module example
 mkdir kernel_module_example
 cd kernel_module_example
  • Create the source code file for this kernel module example: kernel_module_example.c
// SPDX-identifier: GPL-2.0
/*
 * Copyright (C) STMicroelectronics SA 2018
 *
 * Authors: Jean-Christophe Trotin <jean-christophe.trotin@st.com>
 *
 */

#include <linux/module.h>    /* for all kernel modules */
#include <linux/kernel.h>    /* for KERN_INFO */
#include <linux/init.h>      /* for __init and __exit macros */

static int __init kernel_module_example_init(void)
{
	printk(KERN_INFO "Kernel module example: hello world from STMicroelectronics\n");
	return 0;
}

static void __exit kernel_module_example_exit(void)
{
	printk(KERN_INFO "Kernel module example: goodbye from STMicroelectronics\n");
}

module_init(kernel_module_example_init);
module_exit(kernel_module_example_exit);

MODULE_DESCRIPTION("STMicroelectronics simple external out-of-tree Linux kernel module example");
MODULE_AUTHOR("Jean-Christophe Trotin <jean-christophe.trotin@st.com>");
MODULE_LICENSE("GPL v2");
  • Create the makefile for this kernel module example: Makefile
Info white.png Information
All the indentations in a makefile are tabulations
# Makefile for simple external out-of-tree Linux kernel module example

# Object file(s) to be built
obj-m := kernel_module_example.o

# Path to the directory that contains the Linux kernel source code
# and the configuration file (.config)
KERNEL_DIR ?=  $(KERNEL_SRC_PATH)

# Path to the directory that contains the generated objects
DESTDIR ?= $(KERNEL_DIR)/install_artifact

# Path to the directory that contains the source file(s) to compile
PWD := $(shell pwd) 
  
default:
	$(MAKE) -C $(KERNEL_DIR) M=$(PWD) modules

install:
	$(MAKE) -C $(KERNEL_DIR) M=$(PWD) INSTALL_MOD_PATH=$(DESTDIR) modules_install

clean:
	$(MAKE) -C $(KERNEL_DIR) M=$(PWD) clean
  • Cross-compile the kernel module example
 make clean
 make
 make install
  • Go to the Linux kernel build directory
 cd <Linux kernel build directory>
  • The generated kernel module example is in: install_artifact/lib/modules/"<kernel version>"/extra/kernel_module_example.ko
  • Remove the link on "install_artifact/lib/modules/<kernel version>/"
 rm install_artifact/lib/modules/<kernel version>/build
 rm install_artifact/lib/modules/<kernel version>/source
  • Optionally, strip kernel modules (to reduce the size of each kernel modules)
 find . -name "*.ko" | xargs $STRIP --strip-debug --remove-section=.comment --remove-section=.note --preserve-dates
  • Push this kernel module example on board (please check the kernel version "<kernel version>")
 ssh root@<board ip address> mkdir -p /lib/modules/"<kernel version>"/extra
 scp install_artifact/lib/modules/<kernel version>/extra/kernel_module_example.ko root@<board ip address>:/lib/modules/<kernel version>/extra

OR

 scp -r install_artifact/lib/modules/* root@<board ip address>:/lib/modules/
  • Update dependency descriptions for loadable kernel modules, and synchronize the data on disk with memory
 /sbin/depmod -a
 sync
  • Insert the kernel module example into the Linux kernel
 modprobe kernel_module_example
[18167.821725] Kernel module example: hello world from STMicroelectronics
  • Remove the kernel module example from the Linux kernel
 rmmod kernel_module_example
[18180.086722] Kernel module example: goodbye from STMicroelectronics

7 Modifying the U-Boot[edit]

This simple example adds unconditional log information when U-Boot starts. Within the scope of the optee boot chain, U-Boot is used as second stage boot loader (SSBL).

  • Have a look at the U-Boot log information when the board reboots
 reboot
[...]
U-Boot <U-Boot version> (<U-Boot build time flag>)
CPU: <CPU tag marking>
Model: STMicroelectronics <board model details>
Board: stm32mp1 in <boot chain mode> mode (st,<device tree>)
[...]
  • Go to the U-Boot source directory
 cd <U-Boot source directory>

Example:

  • Edit the ./board/st/stm32mp1/stm32mp1.c source file to add a log information in the checkboard function
int checkboard(void)
{
	int ret;
	char *mode;

 	[...]
	log_info("Board: stm32mp1 in %s mode (%s)\n", mode,
		 fdt_compat && fdt_compat_len ? fdt_compat : "");
	log_info("U-Boot simple example\n");
	[...]

	return 0;
}
  • Get the list of supported configurations with the following command
 make -f $PWD/../Makefile.sdk help

FIP images case building (Default)

  • As mentionned in help, OpenSTLinux has activated FIP by default, so the "FIP_artifacts" folder should be specified to allow to update the FIP binary with the new U-Boot binary. When using the "SOURCES-xxxx.tar.gz" from Developer package the "FIP_DEPLOYDIR_ROOT" variable should be set as below:
 export FIP_DEPLOYDIR_ROOT=$PWD/../../FIP_artifacts
  • Note that to keep the original FIP binary, the "FIP_DEPLOYDIR_FIP" variable should be set as below:
 export FIP_DEPLOYDIR_FIP=$PWD/../deploy/fip
  • Cross-compile the U-Boot:
- E.g. for optee boot for STM32MP157F-EV1 More info green.png and STM32MP157F-DK2 More info green.png
 make -f $PWD/../Makefile.sdk all UBOOT_CONFIG=optee UBOOT_DEFCONFIG=stm32mp15_trusted_defconfig UBOOT_BINARY=u-boot.dtb DEVICETREE="stm32mp157f-ev1 stm32mp157f-dk2"
- E.g. for optee boot for STM32MP135F-DK More info green.png
 make -f $PWD/../Makefile.sdk all UBOOT_CONFIG=optee UBOOT_DEFCONFIG=stm32mp13_defconfig UBOOT_BINARY=u-boot.dtb DEVICETREE="stm32mp135f-dk"
  • Go to the directory where the compilation results are stored
 cd $FIP_DEPLOYDIR_FIP
  • Reboot the board, and hit any key to stop in the U-boot shell
 reboot
[...]
Hit any key to stop autoboot:  0 
STM32MP> 
  • Connect a USB cable between the host machine and the board via the USB OTG ports
  • In the U-Boot shell, call the USB mass storage function
STM32MP> ums 0 mmc 0
Info white.png Information
For more information about the usage of U-Boot UMS functionality, see How to use USB mass storage in U-Boot


  • On the host machine, check the partition associated with the secondary stage boot loader that contains the FIP binary (fip-a): sdb5 here
 ls -l /dev/disk/by-partlabel/
total 0
lrwxrwxrwx 1 root root 10 Feb  8 08:57 bootfs -> ../../sdb8
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-a -> ../../sdb5
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-b -> ../../sdb6
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata1 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata2 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Feb  8 08:57 rootfs -> ../../sdb10
lrwxrwxrwx 1 root root 10 Feb  8 08:57 u-boot-env -> ../../sdb7
lrwxrwxrwx 1 root root 10 Feb  8 08:57 userfs -> ../../sdb11
lrwxrwxrwx 1 root root 10 Feb  8 08:57 vendorfs -> ../../sdb9
  • Copy the FIP binary to the dedicated partition
 dd if=fip-<device tree>-<boot chain mode>.bin of=/dev/sdb5 bs=1M conv=fdatasync
  • Reset the U-Boot shell
STM32MP> reset
  • Have a look at the new U-Boot log information when the board reboots
[...]
U-Boot <U-Boot version> (<U-Boot build time flag>)
CPU: <CPU tag marking>
Model: STMicroelectronics <board model details>
Board: stm32mp1 in <boot chain mode> mode (st,<device tree>)
U-Boot simple example
[...]

8 Modifying the TF-A[edit]

This simple example adds unconditional log information when the TF-A starts. Within the scope of the optee boot chain, TF-A is used as first stage boot loader (FSBL).

  • Have a look at the TF-A log information when the board reboots
 reboot
[...]
INFO:    PMIC version = 0x21
INFO:    Reset reason (0x35):
INFO:      Power-on Reset (rst_por)
[...]
  • Go to the TF-A source directory
 cd <TF-A source directory>

Example:

  • Edit the ./plat/st/stm32mp1/bl2_plat_setup.c source file to add a log information in the print_reset_reason function
static void print_reset_reason(void)
{
	[...]
	INFO("Reset reason (0x%x):\n", rstsr);

	INFO("TF-A simple example\n");
	[...]
}
  • Get the list of supported configurations with the following command
 make -f $PWD/../Makefile.sdk help

FIP images case building (Default)

  • Cross-compile the TF-A:
- here microSD card boot for STM32MP157F-EV1 More info green.png and STM32MP157F-DK2 More info green.png
 make -f $PWD/../Makefile.sdk TF_A_DEVICETREE="stm32mp157f-ev1 stm32mp157f-dk2" TF_A_CONFIG="sdcard" ELF_DEBUG_ENABLE='1' stm32
- here microSD card boot for STM32MP135F-DK More info green.png
 make -f $PWD/../Makefile.sdk TF_A_DEVICETREE="stm32mp135f-dk" TF_A_CONFIG="sdcard" ELF_DEBUG_ENABLE='1' stm32
  • Go to the directory in which the compilation results are stored
 cd ../deploy
  • Reboot the board, and hit any key to stop in the U-boot shell
 reboot
[...]
Hit any key to stop autoboot:  0 
STM32MP> 
  • Connect a USB cable between the host machine and the board via the USB OTG ports
  • In the U-Boot shell, call the USB mass storage function
STM32MP> ums 0 mmc 0
Info white.png Information
For more information about the usage of U-Boot UMS functionality, see How to use USB mass storage in U-Boot
  • On the host machine, check the partition associated with the first stage boot loader (fsbl1 and fsbl2 as backup): sdb1 and sdb2 (as backup) here
 ls -l /dev/disk/by-partlabel/
total 0
lrwxrwxrwx 1 root root 10 Feb  8 08:57 bootfs -> ../../sdb8
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-a -> ../../sdb5
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-b -> ../../sdb6
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata1 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata2 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Feb  8 08:57 rootfs -> ../../sdb10
lrwxrwxrwx 1 root root 10 Feb  8 08:57 u-boot-env -> ../../sdb7
lrwxrwxrwx 1 root root 10 Feb  8 08:57 userfs -> ../../sdb11
lrwxrwxrwx 1 root root 10 Feb  8 08:57 vendorfs -> ../../sdb9
  • Copy the TF-A binary to the dedicated partition; to test the new TF-A binary, it might be useful to keep the old TF-A binary in the backup FSBL (fsbl2)
 dd if=tf-a-<device tree>-sdcard.stm32 of=/dev/sdb1 bs=1M conv=fdatasync
Info white.png Information
In case you get a permission denied you can set more permission on /dev/sdb1 : sudo chmod 777 /dev/sdb1
  • Reset the U-Boot shell

In the U-Boot shell, press Ctrl+C prior to get hand back.

STM32MP> reset
  • Have a look at the new TF-A log information when the board reboots
[...]
INFO:    PMIC version = 0x21
INFO:    Reset reason (0x35):
INFO:      Power-on Reset (rst_por)
INFO:    TF-A simple example
[...]
  • Note that according to the modification done on TF-A source code, the FIP binary may need also to be updated. In such case, to cross-compile for selected boot chain with FIP binary update, follow the steps:
  • The "FIP_artifacts" folder should be specified to allow to update the FIP binary with the new U-Boot binary. When using the "SOURCES-xxxx.tar.gz" from Developer package the "FIP_DEPLOYDIR_ROOT" var should be set as below:
 export FIP_DEPLOYDIR_ROOT=$PWD/../../FIP_artifacts
  • To keep the original FIP binary, the "FIP_DEPLOYDIR_FIP" var should be set as below:
 export FIP_DEPLOYDIR_FIP=$PWD/../deploy/fip
  • Launch cross-compilation:
- optee boot for STM32MP157F-EV1 More info green.png and STM32MP157F-DK2 More info green.png:
 make -f $PWD/../Makefile.sdk TF_A_DEVICETREE="stm32mp157f-ev1 stm32mp157f-dk2" TF_A_CONFIG="optee" ELF_DEBUG_ENABLE='1' fip
- optee boot for STM32MP135F-DK More info green.png
 make -f $PWD/../Makefile.sdk TF_A_DEVICETREE="stm32mp135f-dk" TF_A_CONFIG="optee" ELF_DEBUG_ENABLE='1' fip
  • Go to the directory in which the compilation results are stored
 cd $FIP_DEPLOYDIR_FIP
  • Reboot the board, and hit any key to stop in the U-boot shell
 reboot
[...]
Hit any key to stop autoboot:  0 
STM32MP> 
  • Connect a USB cable between the host machine and the board via the USB OTG ports
  • In the U-Boot shell, call the USB mass storage function
STM32MP> ums 0 mmc 0
Info white.png Information
For more information about the usage of U-Boot UMS functionality, see How to use USB mass storage in U-Boot
  • On the host machine, check the partition associated with the secondary stage boot loader that contains the FIP binary (fip-a): sdb5 here
 ls -l /dev/disk/by-partlabel/
total 0
lrwxrwxrwx 1 root root 10 Feb  8 08:57 bootfs -> ../../sdb8
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-a -> ../../sdb5
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-b -> ../../sdb6
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata1 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata2 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Feb  8 08:57 rootfs -> ../../sdb10
lrwxrwxrwx 1 root root 10 Feb  8 08:57 u-boot-env -> ../../sdb7
lrwxrwxrwx 1 root root 10 Feb  8 08:57 userfs -> ../../sdb11
lrwxrwxrwx 1 root root 10 Feb  8 08:57 vendorfs -> ../../sdb9
  • Copy the FIP binary to the dedicated partition
 dd if=fip-<device tree>-<boot chain mode>.bin of=/dev/sdb5 bs=1M conv=fdatasync
Info white.png Information
In case you get a permission denied you can set more permission on /dev/sdb3 : sudo chmod 777 /dev/sdb3
  • Reset the U-Boot shell

In the U-Boot shell, press Ctrl+C prior to get hand back.

STM32MP> reset

9 Modifying the OP-TEE OS[edit]

This simple example adds unconditional log information when the OP-TEE OS starts.

  • Have a look at the OP-TEE OS log information when the board reboots
 reboot
[...]
I/TC: Non-secure external DT found
I/TC: Embedded DTB found
I/TC: OP-TEE version: <OP-TEE OS version> (gcc version <GCC version>) [...]
I/TC: WARNING: This OP-TEE configuration might be insecure!
[...]
  • Go to the OP-TEE OS source directory
 cd <OP-TEE OS source directory>

Example:

  • Edit the ./core/arch/arm/kernel/boot.c source file to add a log information in the __weak boot_init_primary_late function
void __weak boot_init_primary_late(unsigned long fdt)
{
	init_external_dt(fdt);
	[...]

	IMSG("OP-TEE version: %s", core_v_str);
	IMSG("OP-TEE OS simple example");
 	[...]
}
  • Get the list of supported configurations with the following command
 make -f $PWD/../Makefile.sdk help

FIP images case building (Default)

  • As mentionned in help, OpenSTLinux has activated FIP by default, so the "FIP_artifacts" folder should be specified to allow to update the FIP binary with the new OP-TEE OS binary. When using the "SOURCES-xxxx.tar.gz" from Developer package the "FIP_DEPLOYDIR_ROOT" variable should be set as below:
 export FIP_DEPLOYDIR_ROOT=$PWD/../../FIP_artifacts
  • Note that to keep the original FIP binary, the "FIP_DEPLOYDIR_FIP" variable should be set as below:
 export FIP_DEPLOYDIR_FIP=$PWD/../deploy/fip
  • Cross-compile the OP-TEE OS:
- E.g. for optee boot for STM32MP157F-EV1 More info green.png and STM32MP157F-DK2 More info green.png
 make -f $PWD/../Makefile.sdk all CFG_EMBED_DTB_SOURCE_FILE="stm32mp157F-ev1 stm32mp157f-dk2"
- E.g. for optee boot for STM32MP135F-DK More info green.png
 make -f $PWD/../Makefile.sdk all CFG_EMBED_DTB_SOURCE_FILE="stm32mp135f-dk"
  • Go to the directory where the compilation results are stored
 cd $FIP_DEPLOYDIR_FIP
  • Reboot the board, and hit any key to stop in the U-boot shell
 reboot
[...]
Hit any key to stop autoboot:  0 
STM32MP> 
  • Connect a USB cable between the host machine and the board via the USB OTG ports
  • In the U-Boot shell, call the USB mass storage function
STM32MP> ums 0 mmc 0
Info white.png Information
For more information about the usage of U-Boot UMS functionality, see How to use USB mass storage in U-Boot


  • On the host machine, check the partition associated with the secondary stage boot loader that contains the FIP binary (fip-a): sdb5 here
 ls -l /dev/disk/by-partlabel/
total 0
lrwxrwxrwx 1 root root 10 Feb  8 08:57 bootfs -> ../../sdb8
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-a -> ../../sdb5
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-b -> ../../sdb6
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata1 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata2 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Feb  8 08:57 rootfs -> ../../sdb10
lrwxrwxrwx 1 root root 10 Feb  8 08:57 u-boot-env -> ../../sdb7
lrwxrwxrwx 1 root root 10 Feb  8 08:57 userfs -> ../../sdb11
lrwxrwxrwx 1 root root 10 Feb  8 08:57 vendorfs -> ../../sdb9
  • Copy the FIP binary to the dedicated partition
 dd if=fip-<device tree>-<boot chain mode>.bin of=/dev/sdb5 bs=1M conv=fdatasync
  • Reset the U-Boot shell
STM32MP> reset
  • Have a look at the new OP-TEE OS log information when the board reboots
[...]
I/TC: Non-secure external DT found
I/TC: Embedded DTB found
I/TC: OP-TEE version: <OP-TEE OS version> (gcc version <GCC version>) [...]
I/TC: OP-TEE OS simple example
I/TC: WARNING: This OP-TEE configuration might be insecure!
[...]

10 Adding a "hello world" user space example[edit]

Thanks to the OpenSTLinux SDK, it is easy to develop a project outside of the OpenEmbedded build system. This chapter shows how to compile and execute a simple "hello world" example.

10.1 Source code file[edit]

  • Go to the working directory that contains all the source codes (i.e. directory that contains the Linux kernel, Optee-os, TF-A and U-Boot source code directories)
 cd <tag>/sources/arm-<distro>-linux-gnueabi
  • Create a directory for this user space example
 mkdir hello_world_example
 cd hello_world_example
  • Create the source code file for this user space example: hello_world_example.c
// SPDX-identifier: GPL-2.0
/*
 * Copyright (C) STMicroelectronics SA 2018
 *
 * Authors: Jean-Christophe Trotin <jean-christophe.trotin@st.com>
 *
 */

#include <stdio.h>
#include <unistd.h>

int main(int argc, char **argv)
{
	int i =11;

	printf("\nUser space example: hello world from STMicroelectronics\n");
	setbuf(stdout,NULL);
	while (i--) {
		printf("%i ", i);
		sleep(1);
	}
	printf("\nUser space example: goodbye from STMicroelectronics\n");

	return(0);
}

10.2 Cross-compilation[edit]

Three ways to use the OpenSTLinux SDK to cross-compile this user space example are proposed below: (1) command line (2) makefile-based project (3) autotools-based project.

10.2.1 Command line[edit]

This method allows quick cross-compilation of a single-source code file. It applies if the project has only one file.
The cross-development toolchain is associated with the sysroot that contains the header files and libraries needed for generating binaries that run on the target architecture (see SDK for OpenSTLinux distribution#Native and target sysroots).
The sysroot location is specified with the --sysroot option.

The sysroot location must be specified using the --sysroot option. The CC environment variable created by the SDK already includes the --sysroot option that points to the SDK sysroot location.

 echo $CC
arm-ostl-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=<SDK installation directory>/SDK/sysroots/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi
  • Create the directory in which the generated binary is to be stored
 mkdir -p install_artifact install_artifact/usr install_artifact/usr/local install_artifact/usr/local/bin
  • Cross-compile the single source code file for the user space example
 $CC hello_world_example.c -o ./install_artifact/usr/local/bin/hello_world_example

10.2.2 Makefile-based project[edit]

For this method, the cross-toolchain environment variables established by running the cross-toolchain environment setup script are subject to general make rules.
For example, see the following environment variables:

 echo $CC
arm-ostl-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=<SDK installation directory>/SDK/sysroots/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi
 echo $CFLAGS
-O2 -pipe -g -feliminate-unused-debug-types
 echo $LDFLAGS
-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed
 echo $LD
arm-ostl-linux-gnueabi-ld --sysroot=<SDK installation directory>/SDK/sysroots/cortexa7t2hf-neon-vfpv4-ostl-linux-gnueabi
  • Create the makefile for this user space example: Makefile
Info white.png Information
All the indentations in a makefile are tabulations
PROG = hello_world_example
SRCS = hello_world_example.c
OBJS = $(SRCS:.c=.o)

CLEANFILES = $(PROG)
INSTALL_DIR = ./install_artifact/usr/local/bin

# Add / change option in CFLAGS if needed
# CFLAGS += <new option>

$(PROG):  $(OBJS)
	$(CC) $(CFLAGS) -o $(PROG) $(OBJS)

.c.o:
	$(CC) $(CFLAGS) -c $< -o $@

all: $(PROG)
 
 
clean:
	rm -f $(CLEANFILES) $(patsubst %.c,%.o, $(SRCS)) *~

install: $(PROG)
	mkdir -p $(INSTALL_DIR)
	install $(PROG) $(INSTALL_DIR)
  • Cross-compile the project
 make
 make install

10.2.3 Autotools-based project[edit]

This method creates a project based on GNU autotools.

  • Create the makefile for this user space example: Makefile.am
bin_PROGRAMS = hello_world_example
hello_world_example_SOURCES = hello_world_example.c
  • Create the configuration file for this user space example: configure.ac
AC_INIT(hello_world_example,0.1)
AM_INIT_AUTOMAKE([foreign])
AC_PROG_CC
AC_PROG_INSTALL
AC_OUTPUT(Makefile)
  • Generate the local aclocal.m4 files and create the configure script
 aclocal
 autoconf
  • Generate the files needed by GNU coding standards (for compliance)
 touch NEWS README AUTHORS ChangeLog
  • Generate the links towards SDK scripts
 automake -a
                       
  • Cross-compile the project
 ./configure ${CONFIGURE_FLAGS}
 make
 make install DESTDIR=./install_artifact

10.3 Deploy and execute on board[edit]

  • Check that the generated binary for this user space example is in: ./install_artifact/usr/local/bin/hello_world_example
  • Push this binary onto the board
 scp -r install_artifact/* root@<board ip address>:/
  • Execute this user space example
 cd /usr/local/bin
 ./hello_world_example 

User space example: hello world from STMicroelectronics
10 9 8 7 6 5 4 3 2 1 0 
User space example: goodbye from STMicroelectronics

11 Tips[edit]

11.1 Creating a mounting point[edit]

The objective is to create a mounting point for the boot file system (bootfs partition)

  • Find the partition label associated with the boot file system
 ls -l /dev/disk/by-partlabel/
total 0
lrwxrwxrwx 1 root root 10 Feb  8 08:57 bootfs -> ../../mmcblk0p8
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-a -> ../../mmcblk0p5
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fip-b -> ../../mmcblk0p6
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl1 -> ../../mmcblk0p1
lrwxrwxrwx 1 root root 10 Feb  8 08:57 fsbl2 -> ../../mmcblk0p2
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata1 -> ../../mmcblk0p3
lrwxrwxrwx 1 root root 10 Feb  8 08:57 metadata2 -> ../../mmcblk0p4
lrwxrwxrwx 1 root root 10 Feb  8 08:57 rootfs -> ../../mmcblk0p10
lrwxrwxrwx 1 root root 10 Feb  8 08:57 u-boot-env -> ../../mmcblk0p7
lrwxrwxrwx 1 root root 10 Feb  8 08:57 userfs -> ../../mmcblk0p11
lrwxrwxrwx 1 root root 10 Feb  8 08:57 vendorfs -> ../../mmcblk0p9
  • Attach the boot file system found under /dev/mmcblk0p8 in the directory /boot
 mount /dev/mmcblk0p8 /boot