How to debug OP-TEE

Revision as of 10:58, 20 February 2019 by Registered User (→‎Debug with GDB: fix JPR comment)

Template:ArticleMainWriter Template:ArticleProposedVersion

1. Purpose[edit source]

This article explains how to debug the OP-TEE secure world binaries.
These debug information are specifically linked to the CPU secure state (Arm® TrustZone®).

The OP-TEE secure world binaries include OP-TEE core (privilege firmware) and OP-TEE trusted applications and libraries (user space context):

  • There are two main ways to debug OP-TEE core: using embedded traces or using JTAG/SWD to access the secure world.
    The focus here is on the solution integrated in OpenSTLinux: debug over GDB (ST-LINK or JTAG/SWD based).
  • The OP-TEE trusted applications and libraries provide debug support relying on embedded traces only.
    It is not recommended to debug the secure userland binaries through JTAG/SWD resources and OP-TEE does not provide means to embed a GDB server in the secure world.
Warning white.png Warning
This article focuses on OP-TEE debug.
Refer to STM32MP1 Platform trace and debug environment overview article for more generic information.

2. Debugging OP-TEE core[edit source]

2.1. OP-TEE Version number[edit source]

The starting point for debugging OP-TEE core is to identify the OP-TEE version embedded in the target. A version identifier is displayed on the console with the following format:

I/TC: OP-TEE version:<tag> #<buildcount> <date> <arch>

I.e:

I/TC: OP-TEE version: openstlinux-19-01-11-10-g56ef3b0 #1 Wed Jan 30 09:12:56 UTC 2019 arm

2.2. Embedded assertions[edit source]

OP-TEE core can embed debug assertions that panic the system when tested condition is not met. Embedded assertions are implemented using function assert(), i.e in the following code snippet, the system will panic if argument1 is negative while function returns the decremented value of the input argument:

static int increment_argument(int argument)
{
   assert(argument > 0);
   return argument - 1;
}

Assertions are embedded or not upon configuration directive CFG_TEE_CORE_DEBUG={n|y} when OP-TEE core is built.

2.3. Debug with traces[edit source]

OP-TEE provides trace messages support and a configurable trace level build directive CFG_TEE_CORE_LOG_LEVEL={0|1|2|3|4}. This directive defines the trace levels that are embedded inside the firmware and output through the OP-TEE console. A low value reduces the memory footprint of the firmware and increase its runtime performances.

Value Name Description with related macros
0 - All trace messages are disabled
1 Error trace level Only non-tagged and error trace messages are embedded: MSG(), MSG_RAW(), EMSG(), EMSG_RAW()
2 Info trace level + info trace messages: IMSG(), IMSG_RAW()
3 Debug trace level + debug trace messages: DMSG(), DMSG_RAW()
4 Flow trace level + flow trace messages: FMSG(), FMSG_RAW()
Warning white.png Warning
All debug support cannot be enabled together due to the internal memory size constraint. If required, change some DMSG() into IMSG().

Traces and errors are available on the console defined in the chosen node of the device tree by the stdout-path property:

chosen {
        stdout-path = "serial0:115200n8";
};

More information about OP-TEE build and update is available in the STM32MP15 OP-TEE article.

2.4. Stack unwind support[edit source]

OP-TEE core can trace the execution backtrace when its panics. The execution backtrace allows one to analyse the execution call sequence that led to the panic. OP-TEE OS provides tools to analyse such backtrace based on the OP-TEE core ELF file generated at built time.

Backtrace unwind is embedded or not upon configuration directive CFG_UNWIND={y|n} Note that backtrace unwind increases the size of the OP-TEE core firmware. When OP-TEE core executes from a small secure RAM, enabling stack unwind penalize the OP-TEE core performances.

More information are available from OP-TEE abort dumps documentation[1].

2.5. Debug with GDB[edit source]

The Debug OpenSTLinux BSP Components with GDB article describes how to set up GDB / OpenOCD environment.
One can debug OP-TEE through JTAG/SWD using an ST-LINK or the JTAG/SWD output, depending on the target board.

Note that when OP-TEE core executes in a small secure memory with support of its pager (case build directive CFG_WITH_PAGER=y), it is highly recommended to use hardware breakpoint rather than software breakpoint. Since most of the OP-TEE core instructions dynamically loaded into the small secure memory, loading a software breakpoint is likely to be discarded when OP-TEE pager wipes memory content to load other OP-TEE core pages.

When OP-TEE executes in the large main memory (case build directive CFG_WITH_PAGER is disabled), all OP-TEE core resources are resident. In this case, one can use as well hardware as software breakpoints without any issues.

2.5.1. Debug boot sequence[edit source]

Load symbols to the target offset:

(gdb) add-symbol-file <path_to_build_folder>/tee.elf <load_address>

OP-TEE load address is available from the generated tee-init_load_addr.txt file.
It can also be found in the generated tee.map file:

...
Linker script and memory map

                0x000000002ffc0000                . = 0x2ffc0000
                0x0000000000000001                ASSERT (0x1, text start should align to 32bytes)
                0x000000002ffc0000                __text_start = .
                0x000000002ffc0000                __flatmap_unpg_rx_start = ((__text_start / 0x1000) * 0x1000)

.text           0x000000002ffc0000     0xc538
 *(SORT_BY_ALIGNMENT(.text._start))
 .text._start   0x000000002ffc0000       0x98 out/stm32mp157c-ev1/core/arch/arm/kernel/generic_entry_a32.o
                0x000000002ffc0000                _start  -> OP-TEE Load address
...

In this example, OP-TEE load address is 0x2ffc0000.

One can load all OP-TEE core symbols:

(gdb) add-symbol-file <path_to_build_folder>/tee.elf 0x2ffc0000

One can set a hardware breakpoint at OP-TEE core entry point and reset the target:

(gdb) hb _start
(gdb) monitor reset halt

2.5.2. Debugging during runtime execution[edit source]

Once U-Boot or the Linux kernel is running, one cannot access secure memory or regions, but can break, set a hardware breakpoint on OP-TEE service handler. GDB will break once it has switched into the secure world and reached the break instruction. Once halted, GDB can access secure resources as peripheral interfaces or memories. For example, one can break into U-Boot use the following GDB instructions:

(gdb) hb stm32_sip_service
(gdb) continue

On the first service call occurrence GDB breaks into the stm32_sip_service() entry in OP-TEE core.

3. Debugging OP-TEE Trusted Applications[edit source]

3.1. Embedded assertions[edit source]

OP-TEE trusted applications can embed assertions as well as the OP-TEE core as described above.
Assertion in trusted applications are embedded upon configuration directive CFG_TEE_CORE_DEBUG={y|n} when OP-TEE OS package is built.

3.2. Debug with traces[edit source]

OP-TEE trusted applications can embed trace messages using the same macros as OP-TEE core (EMSG() and friends described above).
The build directive that sets the trace level for a trusted application is CFG_TEE_TA_LOG_LEVEL={0|1|2|3|4}.
The level values match those described for CFG_TEE_CORE_LOG_LEVEL in section above.

3.3. Stack unwind support[edit source]

When an OP-TEE trusted application panics, it output may output backtrace messages on OP-TEE console as the OP-TEE core.
This backtrace messages are generated by the OP-TEE core which shall be built with CFG_UNWIND=y as described in section above.

4. References[edit source]