How to add a new parallel or serial NAND flash memory device in Yocto

This article explains step-by-step how to configure, create, and populate the serial NAND partitions with the right size, in order to boot from a serial NAND flash memory device (UBI format).

1 Introduction

STM32MP157x-EV1 boards can boot from different storage devices (such as eMMC, SD card, parallel NAND flash memory or NOR flash memory).

Other external storage devices, present, for example, on the expansion board connected to STM32MP157x-EV1 board, can be chosen to boot from. In order to achieve this, some parameters in Yocto recipes must be modified to generate a functional image.

For NAND flash memory devices, UBIFS is the only filesystem supported by OpenSTLinux platform. This article explains step-by-step how to configure, create, and populate the serial NAND partitions with the right size, in order to boot from a serial NAND flash memory device (UBI format).

For demonstration purposes, this article takes the example of a serial NAND flash memory device, but it can be transposed to any NAND device.

To help you to understand this complex article, it is recommended to open, read, and analyze also the file available in the following OpenSTLinux distribution directory conf/machine/include.

The first step is to calculate the right size of serial NAND partitions. Then, based on these results, the UBIFS volume is created and populated.

2 How to calculate the size of serial NAND flash memory partitions

This chapter aims at describing how to configure the serial NAND flash memory UBI partitioning to enable the board to boot from it.

Several partitions are defined for STM32MP157x-EV1 boards. The list can be found in the Flash partitions article.

In the whole article, the serial NAND flash memory, MT29F2G01ABA, is taken as an example.
This memory device has a size of 256 Mbytes. Stored partitions should be sized as shown in the table below. FBBL1, SSBL1, and SSBL2 sizes already take into account the default OpenSTLinux constraints, and are defined in a Linux^® layer. The user can fine-tune these sizes to better fit his/her product, in st-machine-common-stm32mp.inc file:

Partition	Type	Content	Size
FSBL1	raw	fsbl binaries	2 Mbytes
SSBL1	raw	FIP binary	4 Mbytes
SSBL2	raw	FIP binary	4 Mbytes
UBI	ubifs	multi volume UBI	UBI size (in Mbytes)

The memory storage device stores all the boot stages as well as the UBI filesystem.
Boot stages include FSBL (first stage bootloader) and SSBL (second stage bootloader). The SSBL can be included into the FIP and is duplicated on the memory device.

The UBI size corresponding to this memory device is:

UBI size =  SPI NAND size - (FSBL1 + SSBL1 + SSBL2) 
UBI size for the example given =  256 Mbytes - (2 + 4 +4) = 246 Mbytes

The following software parts must be stored in the multivolume UBI:

• uboot_config,
• uboot_config_r,
• bootfs,
• vendorfs,
• rootfs
• userfs.

Volume	Type	Content	Size
uboot_config	UBI	empty	256 Kbytes
uboot_config_r	UBI	empty	256 Kbytes
boot	ubifs	bootfs	64 Mbytes
vendorfs	ubifs	vendorfs	16 Mbytes
rootfs	ubifs	rootfs	Maximum available space
userfs	ubifs	userfs	16 Mbytes

To take into account the small storage capacity of this memory device, the size of the userfs partition must be reduced from 128 Mbytes (default value in OpenSTLinux) to 16 Mbytes, in order to free more space for the rootfs partition.

To generate a UBI filesystem, some parameters must be set according to the storage device. The next chapters explain how to calculate these parameters.

2.1 Storage device characteristics

In the serial NAND flash memory MT29F2G01ABA datasheet, the memory sizes are defined as follows:

• Memory device size: 256 Mbytes
• 2048 blocks
  • Block size: 128 Kbytes
  • Page size: 2048 bytes

2.2 Preamble : Variable naming

The STM32MP157x-EV1 provided layers supports several kinds of storage devices. Some share the same partition configuration regarding the size and partition type (for example eMMC and SD card), while others are different (for example parallel NAND flash memory devices). To be able to define and build everything at one time, the variables can be overloaded with a suffix.
It has been arbitrarily decided to use the following taxonomy: suffix _<memory device type>_<page size in Kbytes>_<block size expressed in Kbytes> .
In the chapters below, all the MT29F2G01ABA variables are named _spinand_2_128, for example EXTRA_UBIFS_SIZE_spinand_2_128.

2.3 Calculating the maximum size of the roofs partition

The UBI is made of four partitions (bootfs, vendorfs, rootfs, userfs), two u-boot configuration parameters storage area (u-boot_config, u-boot_config_r), and some UBI overhead. The UBI overhead corresponds to some extra memory space, reserved to NAND flash memory driver, and which purpose is to handle the bad blocks.
In addition, for each UBIFS partition, this serial NAND flash memory uses several "extra blocks". The experience shows that each UBIFS can require an additional block. After booting the board, the final partition size is smaller than expected.

To generate the UBI layer, the exact size of each partition must be known. This means that, contrary to SD-card management, the rootfs size must be defined accurately to prepare NAND memory devices.

The maximal rootfs size can be calculated using the following formula: Max rootfs size = Multivolume UBI - (uboot_config + uboot_config_r + boots + vendorfs + userfs + UBI Overhead + FilesystemNumber * EXTRA_UBIFS_SIZE) where FilesystemNumber = 4 EXTRA_UBIFS_SIZE = size of all extra blocks

Below an explanation on how to calculate EXTRA_UBIFS_SIZE and the UBI overhead.

2.3.1 Defining EXTRA_UBIFS_SIZE value

Generally speaking, when the UBI filesystem is generated, its size does not match the expected one, and it is smaller than expected. The size difference is almost the same on all partitions (such as users, rootfs).

To obtain the expected size for each filesystem (serfs, rootfs, ...), some extra blocks must be added to this expected size.
The number of extra blocks can be empirically determined.
The way to proceed is to define no extra blocks, generate the TSV, and flash it. Check the size of the filesystem, then restart to obtain the right filesystem size. See example below:

In st-machine-common-stm32mp.inc, add or modify the EXTRA_UBIFS_SIZE parameter and set it to 0:

 EXTRA_UBIFS_SIZE_spinand_2_128 = "0"

Then generate the image and flash it.
Once the system has started, mount the partitions and compare their size with the requested size. The Bootfs partition size is set in st-machine-common-stm32mp.inc to 64 Mbytes (67,108,864 bytes).

• bootfs partition

The Bootfs partition requested size is set in st-machine-common-stm32mp.inc to 64 Mbytes (67,108,864 bytes).

   mkdir boot; mount -t ubifs ubi0:boot boot
 [  308.695504] UBIFS (ubi0:2): Mounting in unauthenticated mode
 [  308.700230] UBIFS (ubi0:2): background thread "ubifs_bgt0_2" started, PID 493
 [  308.723561] UBIFS (ubi0:2): start fixing up free space
 [  320.387422] UBIFS (ubi0:2): free space fixup complete
 [  320.404941] UBIFS (ubi0:2): UBIFS: mounted UBI device 0, volume 2, name "boot"
 [  320.410797] UBIFS (ubi0:2): LEB size: 126976 bytes (124 Kbytes), min./max. I/O unit sizes: 2048 bytes/2048 bytes
 [  320.420778] UBIFS (ubi0:2): FS size: 65773568 bytes (62 Mbytes, 518 LEBs), journal size 9023488 bytes (8 Mbytes, 72 LEBs)
 [  320.431222] UBIFS (ubi0:2): reserved for root: 0 bytes (0 Kbyte)
 [  320.437149] UBIFS (ubi0:2): media format: w4/r0 (latest is w5/r0), UUID B9AD3E8C-464F-4AD7-A9EA-C5BCB2AF0C4E, small LPT model

The above log shows that the booths partition size is only 65,773,568 bytes bytes. Additional space must be added to the bootfs partition size. This extra space is called "extra blocks" because it is defined as a number of blocks.
The formula to compute the number of extra blocks is:

extra blocks = (expected size in bytes - real size in bytes)/ memory block size in bytes
extra blocks for the eg = (67,108,864 bytes -  65,773,568 bytes)/ (128*1024) = 10.1875

• vendorfs partition

The vendorfs partition requested size is set in st-machine-common-stm32mp.inc to 16 Mbytes (16,777,216 bytes).

   mkdir vendorfs; mount -t ubifs ubi0:vendorfs vendorfs
 [  496.605692] UBIFS (ubi0:3): Mounting in unauthenticated mode
 [  496.610389] UBIFS (ubi0:3): background thread "ubifs_bgt0_3" started, PID 518
 [  496.627244] UBIFS (ubi0:3): start fixing up free space
 [  497.105173] UBIFS (ubi0:3): free space fixup complete
 [  497.122778] UBIFS (ubi0:3): UBIFS: mounted UBI device 0, volume 3, name "vendorfs"
 [  497.129033] UBIFS (ubi0:3): LEB size: 126976 bytes (124 Kbytes), min./max. I/O unit sizes: 2048 bytes/2048 bytes
 [  497.139010] UBIFS (ubi0:3): FS size: 15491072 bytes (14 Mbytes, 122 LEBs), journal size 9023488 bytes (8 Mbytes, 72 LEBs)
 [  497.149456] UBIFS (ubi0:3): reserved for root: 0 bytes (0 Kbyte)
 [  497.155237] UBIFS (ubi0:3): media format: w4/r0 (latest is w5/r0), UUID A506F102-49C4-413C-9ECA-484E04F311D8, small LPT model

The above log shows that the vendorfs partition size is only 15,491,072 bytes . Additional space must be added to the vendorfs partition size. This extra space is called "extra blocks" because it is defined as a number of blocks.
The formula to compute the number of extra blocks is:

extra blocks = (expected size in bytes - real size in bytes)/ memory block size in bytes
extra blocks for the eg = (16,777,216 bytes  -  15,491,072 bytes)/ (128*1024) = 9.8125

The same computation can be done on the rootfs partition to ensure that the number of extra blocks is correct.

Regarding the userfs partition, the number of extra blocks is more difficult to determine since the partition is extended when the board boots.

The EXTRA_UBIFS_SIZE parameter is common to all partitions and must be an integer. The EXTRA_UBIFS_SIZE must consequently be rounded to the smallest integer superior to the highest value between 9.81 and 10.19. As a result, for each partition, 11 extra blocks must be added:

Then, the parameter EXTRA_UBIFS_SIZE_spinand_2_128 to 11 extrablocks multiplied by the size of a block (128 Kbytes).

For OpenSTLinux distribution, the EXTRA_UBIFS_SIZE is then overloaded by EXTRA_UBIFS_SIZE_spinand_2_128

EXTRA_UBIFS_SIZE = ExtraBlockNumber * block size = 11 * 128 Kbytes = "1408"

This parameter can be defined in file st-machine-common-stm32mp.inc

 EXTRA_UBIFS_SIZE_spinand_2_128 = "1408"

2.3.2 Calculating the UBI overhead size

According to Linux MTD documentation, the UBI overhead is defined by the foliowng formula: 

UBI overhead = (B - BB + 4) * SP + O * (P - B - 4) with
* W - total number of physical erase blocks on the flash memory chip (NB: the entire chip, not the MTD partition);
* P - total number of physical erase blocks on the MTD partition;
* SP - physical erase block size;
* SL - logical erase block size;
* BB - number of bad blocks on the MTD partition;
* BR - number of PEBs reserved for bad PEB handling (it is 20 * W/1024 for NAND by default, and 0 for NOR and other flash memory types, which do not have bad PEBs);
* B - MAX(BR,BB);
* O - the overhead related to storing EC and VID headers in bytes, that is O = SP - SL.

The UBI overhead corresponding to the NAND flash memory described in this article is (20*2048/1024 + 4) * 128 Kbytes + (128 Kbytes - 124 Kbytes) * (246 Mbytes/256 Kbytes - 20*2048/1024 - 4) = 9392 Kbytes

2.3.1 Calculating the maximum rootfs size

As already explained above, the formula to calculate the maximal rootfs size is the following:

Max rootfs size = Multivolume UBI - (uboot_config + uboot_config_r + bootfs + vendorfs + userfs + UBI Overhead + PartitionsNumber * ExtraBlockNumber * Block size ) 

The maximum rootfs size corresponding to the NAND flash memory described in this article is 138064 Kbytes

In Yocto, the partition size is a expressed in Kbytes, which means that the previously calculated size must be divided by 1024, and only the integer part has to be kept. Then multiply the value by 1024:

rootfs size = trunc(Max rootfs size/ 1024)*1024 

For the MT29F2G01ABA flash memory: trunc(138064 / 1024)*1024 = 137216 . So, in the machine configuration, STM32MP_ROOTFS_SIZE can be written like this:

STM32MP_ROOTFS_SIZE_spinand_2_128 ?= "137216"

Do not forget to specify the size of the userfs in the configuration files.

STM32MP_USERFS_SIZE_spinand_2_128 ?= "16384"

2.3.2 Configuring the partition sizes into Yocto layer

The default values of all the partition sizes are defined in st-machine-common-stm32mp.inc:

STM32MP_BOOTFS_SIZE       ?= "65536"
STM32MP_USERFS_SIZE       ?= "131072"
STM32MP_VENDORFS_SIZE       ?= "16384"

The userfs size for the MT29F2G01ABA can be set in the same file: STM32MP_USERFS_SIZE_spinand_2_128 ?= "16384"

This default parameter (STM32MP_USERFS_SIZE ) is overloaded by this new parameter STM32MP_USERFS_SIZE_spinand_2_128.

3 Creating the MultiUBI volume

The OpenSTLinux image generated by ST embeds several UBIFS partitions. This is called the multiubi in ST Yocto configuration files. Only one UBI file can be downloaded into the board. This file must contain all volumes (uboot_config, uboot_config_r, boot, rootfs, vendorfs, userfs).

3.1 Generating automatically UBI and multiUBI volumes

The UBI volume consists of a set of ubifs partitions and some binary files. The ubifs partitions used for creating the UBI volume are defined in conf/machine/include/st-machine-common-stm32mp.inc.

Ubifs partitions are generated in files that must be generated by the command "mkfs.ubifs" before creating the UBI volume.

The UBI volume is generated by using the ubinize command. This tool is a community development. Refer to manpage for details.

The ubinize and "mkfs.ubifs" commands are executed by Yocto during the image generation process. The machine must be configured before (see chapter #Calculate sizes of Serial NAND partitions. The following declarations must be added as follows in the configuration file (in st-machine-common-stm32mp.inc):

UBINIZE_ARGS_spinand_2_128 = "--min-io-size <page size> --peb-size <block size>"
MKUBIFS_ARGS_spinand_2_128 = "--min-io-size  <page size> --leb-size <LEBSize> --max-leb-cnt  <BlockSize> --space-fixup"
where  BlockSize defined in NAND datasheet and LEBSize as explained below.

LEBSize value: UBI uses subpages to reduce flash memory space overhead. This overhead is reduced only if subpages can be used. The MT29F2G01ABA does not have subpages: UBI puts the VID header (volume identifier header) at a physical offset of 2048 (value defined in standard UBI) and the LEB size consequently becomes 124 Kbytes (128 Kbytes minus one NAND page to store the EC header (erase counter header) and minus another NAND page for the VID header).

For the MT29F2G01ABA:

UBINIZE_ARGS_spinand_2_128 = "--min-io-size 2048 --peb-size 128 Kbytes"
MKUBIFS_ARGS_spinand_2_128 = "--min-io-size 2048 --leb-size 126976 --max-leb-cnt 2048 --space-fixup"

3.2 Generating manually UBI and multiUBI volumes

3.2.1 Launching manually mkfs.ubifs command to generate the ubifs partition files

This chapter explains how to launch manually the mkfs.ubifs command. It is very useful to check that the parameters calculated above are well defined to generate the right UBI volume.

The UBI filesystem is generated by using the mkfs.ubifs command. This tool can be found in the MTD utils repository (mkfs.ubifs subdirectory), http://git.infradead.org/mtd-utils.git

This command must be used as follows:

  mkfs.ubifs -r <rootfs>
     -o <output_FILE> \
     --min-io-size <io_size> \
     --leb-size <leb_size> \
     --max-leb-cnt <count> \
     --space-fixup

Parameters:

• -r <rootfs>: the content root directory of the UBIFS image is identical to the <rootfs> directory after the partition generation.

• -o <output_FILE>: output file.

• --min-io-size (or -m): this is the minimum input/output unit size of the memory storage device. The value of the parameter depends on the memory storage device type. For the MT29F2G01ABA, it is equal to the page size, that is 2048 bytes.

• --leb-size (or -e): this parameter is the logical erase block size. It is different from the physical erase block size of the memory storage device. The value depends on the memory storage device. For the MT29F2G01ABA, this parameter is equal to 128K-2*2048 = 124 Kbytes = 126976.

• --max-leb-cnt (or -c): the value depends on the UBI filesystem and memory storage device type. It defines the maximum filesystem size (in erase blocks). For example, if the parameter is 200 LEB, the resulting image can be set to the smaller UBI volume and mounted. However, if the image is set to a larger UBI volume, the filesystem takes only the first 200 LEBs, and the rest of the volume is wasted.

For better understanding of parameters, refer to the UBIFS home page.

The mkfs.ubifs command is executed by Yocto during the image generation process. You only need to configure the parameters into the machine configuration file (for example conf/machine/include/st-machine-common-stm32mp.inc):

 MKUBIFS_ARGS_spinand_2_128 = "--min-io-size 2048 --leb-size 126976 --max-leb-cnt 2048 --space-fixup"

3.2.2 Launching manually the ubinize command to generate the UBI volume

This chapter explains how to launch manually the ubinize command. It is very useful to check that the parameters calculated above are well defined to generate the right UBI volume.

In man page, a lot of options are defined, but only a few are used by Yocto:

  ubinize -o <output_FILE> \
     --min-io-size <io_size> \
     --peb-size <peb_size> \
     <ini-file>

Parameters:

• --min-io-size (or -m): this is the same parameter as the one used for mkfs.ubifs. For the MT29F2G01ABA flash memory, it is equal to the page size, that is 2048 bytes.

• --peb-size (or -p): this parameter depends on the flash memory type. This is the size of the physical erase block in bytes, kilobytes (Kbytes) or megabytes (Mbytes). For the MT29F2G01ABA it is equal to '128 Kbytes.

• <ini-file>: the input configuration <ini-file> describes all the volumes that have to be included in the output UBI image. Each volume is described in a dedicated section, which may be named arbitrarily. Each section consists of "key=value" pairs. For example:

 [ubifs]
 mode=ubi
 image=./my_rootfs.ubifs
 vol_id=0
 vol_type=dynamic
 vol_name=my_rootfs
 vol_size=128 Mbytes

The <ini-file> is created by Yocto during the image generation process. In the considered example, it describes several partitions.

 [uboot_config]
 mode=ubi
 vol_id=0
 vol_type=dynamic
 vol_name=uboot_config
 vol_size=256 Kbytes
 [uboot_config_r]
 mode=ubi
 vol_id=1
 vol_type=dynamic
 vol_name=uboot_config_r
 vol_size=256 KbytesiB
 [st-image-bootfs-stm32mp-valid-stm32mp1]
 mode=ubi
 image=./st-image-bootfs-stm32mp-valid-stm32mp1.bootfs.ubifs
 vol_id=2
 vol_type=dynamic
 vol_name=boot
 vol_size=66944 Kbytes
 [st-image-vendorfs-stm32mp-valid-stm32mp1]
 mode=ubi
 image=./st-image-vendorfs-stm32mp-valid-stm32mp1.vendorfs.ubifs
 vol_id=3
 vol_type=dynamic
 vol_name=vendorfs
 vol_size=17792 Kbytes
 [st-image-core-stm32mp-valid-stm32mp1]
 mode=ubi
 image=./st-image-core-stm32mp-valid-stm32mp1.rootfs.ubifs
 vol_id=4
 vol_type=dynamic
 vol_name=rootfs
 vol_size=138624 Kbytes
 [st-image-userfs-stm32mp-valid-stm32mp1]
 mode=ubi
 image=./st-image-userfs-stm32mp-valid-stm32mp1.userfs.ubifs
 vol_id=5
 vol_type=dynamic
 vol_name=userfs
 vol_flags=autoresize

4 How to improve the boot sequence for NAND memory device

During the boot process, TF-A parses the flash memory device to find bootstage binaries. To avoid parsing the whole flash memory device, a default flash offset (also known as MTD offset) is defined, depending on the flash memory type and boot scheme.

• FIP

#ifndef STM32MP_NAND_FIP_OFFSET
#define STM32MP_NAND_FIP_OFFSET		U(0x00200000)
#endif

• Legacy

 #ifndef STM32MP_NAND_BASE_OFFSET
 #define STM32MP_NAND_BASE_OFFSET        U(0x00200000)
 #endif

These default values may be not suitable for the flash memory, resulting in a timeout during the boot. To avoid the timeout, it is possible to specify the start offset.
To know the offset value, generate the tsv and retrieve the offset value for the first FIP binary.

 #Opt    Id      Name    Type    IP      Offset  Binary
 -       0x01    fsbl1-boot      Binary  none    0x0     arm-trusted-firmware/tf-a-stm32mp157f-ev1-usb.stm32
 -       0x03    fip-boot        Binary  none    0x0     fip/fip-stm32mp157f-ev1-trusted.bin
 P       0x04    fsbl1   Binary(2)       nand0   0x00000000      arm-trusted-firmware/tf-a-stm32mp157f-ev1-nand.stm32
 P       0x05    fip     Binary  nand0   0x00200000      fip/fip-stm32mp157f-ev1-trusted.bin
 P       0x06    fip2    Binary  nand0   0x00600000      fip/fip-stm32mp157f-ev1-trusted.bin
 P       0x10    ubifs   System  nand0   0x00A00000      st-image-core-stm32mp-valid-stm32mp1_nand_4_256_multivolume.ubi

Then add this value to your machine configuration

 TF_A_MTD_START_OFFSET_SPINAND = "0x00200000"

If you use the SDK to build TF-A, add STM32MP_FORCE_MTD_START_OFFSET=0x00200000 in the command line.

Unsorted block images

former spelling for e•MMC ('e' in italic)

SD memory card (https://www.sdcard.org)

Linux^® is a registered trademark of Linus Torvalds.

Firmware Image Package is a packaging format used by TF-A

First Stage Boot Loader

Second Stage Boot Loader

Serial Peripheral Interface

Secure digital

Logical eraseblocks

universally unique identifier (https://en.wikipedia.org/wiki/Universally_unique_identifier)

Memory Technology Device

Trusted Firmware for Arm^® Cortex^®-A

Unsorted block images

former spelling for e•MMC ('e' in italic)

SD memory card (https://www.sdcard.org)

Linux^® is a registered trademark of Linus Torvalds.

Firmware Image Package is a packaging format used by TF-A

First Stage Boot Loader

Second Stage Boot Loader

Serial Peripheral Interface

Secure digital

Logical eraseblocks

universally unique identifier (https://en.wikipedia.org/wiki/Universally_unique_identifier)

Memory Technology Device

Trusted Firmware for Arm^® Cortex^®-A

MultimediaCard

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Cortex^®