|
Coming soon |
1. LC3 Codec
The LC3 codec is an algorithm allowing to compress audio data for transmitting over the air.
This codec must be supported by any application build over the Generic Audio Framework and BLE 5.2 isochronous feature. It runs channel independently, has a complexity similar to Opus, and can be used for either voice and music with a better quality.
1.1. LC3 configuration
BLE profile configures an LC3 session defined by:
- The sampling frequency Fs: 8, 16, 24, 32 ,44.1 or 48 kHz
- The frame duration Nms: 7.5 or 10ms
The frame duration varies a little with 44.1 kHz, refers to LC3 specification [1] for details
Then it also configures each channels referring to the session with additional information:
- The mode : encoding or decoding
- The PCM sample width : 16, 24 or 32 bits
- The bitrate within a list of recommended bitrates
The LC3 supports bitrate updates, but BLE profiles don't use this feature
Also, the ST LC3 codec embed a Packet Loss Concealment algorithm based on the annex B of the LC3 specification. This algorithm ensure signal continuity and reduced glitches in cases of corrupted/missing packet and is triggered by either an external indicator BFI or an internal frame analysis.
At the encoder side, the data flow is:
- Data Input
- PCM signal buffered into an Nf samples per channel, this size is linked to the frame duration and frequency.
- For example, 10ms frame at 32kHz leads to 320 samples
- Data Output
- An encoded buffer per frame per channel of size Nbytes, this size is directly linked to the bitrate and is within the range of 20 to 400 bytes.
- In our example, 64kbps leads to 80 bytes, so compression factor of 8 for 16bits/sample
1.2. LC3 requirement
Our LC3 implementation is based of floating point numbers and needs to access the FPU for fast computation as well as the support of DSP instructions. Also, since it's an heavy process, it is recommenced to use high core clock with Instruction Cache enabled. The library provided is compiled for cortex M33 with speed optimization.
Provided by the upper layer, the following RAM must be allocated for running the full feature LC3:
- 604 bytes per session
- 4820 bytes per encoding channel
- 8340 bytes per decoding channel
Also, at least 7000 bytes are allocated by the encoder over the stack, or 4700 bytes if the only decoder is used.
The flash footprint of the full codec represents around 120 Kbytes and can be reduced at the linking time if only one mode is used.
Here is an overview of the CPU load measured :
This LC3 implementation is SIG certified with the QDID : XXXXX
2. Audio path architecture
2.1. Overview
The Bluetooth specification allows various ISO data path. While the legacy path use the HCI interface, a vendor specific data path allows a better handling of processes and latencies. This architecture puts the audio codec bellow the HCI interface that can be configured using standard HCI commands.
ISO data exchanged with the link layer are called Media packets and can be either, an encoded frame or a raw frame (mode transparent), several channels multiplexed and/or several blocks concatenated. This format is defined in LTV structures and are given through HCI commands.
2.2. Clock synchronization
2.2.1. Generalities
The Bluetooth specification says when use unframed PDU, the upper layer must synchronize generation of its data to the effective transport timings. In BLE, transports timings are defined by the link layer master's sleep clock, and upper layers (including audio peripheral) from both slave and master side have to synchronize to that clock. This statement has also sense when using framed PDU for avoiding glitch generation and having the same data rate from the audio source to the audio sink.
The most constraining role is BLE slave, meaning the device has to synchronize it's audio peripheral clock to the remote link layer.
Beeing BLE master brings less constraint since we only need to synchronize to a local clock. If power consumption is not a deal, the sleep clock can directly be taken from the active clock.
However, being slave on the audio peripheral brings a constraint impossible to respect.
Note that it is not recommended to run BLE audio based on a non accurate sleep clock like LSI.
| RF Role | Audio Role | Sleep Clock Source | Synchronization method |
|---|---|---|---|
| Master | Master | LSE | PLL Loop control |
| HSE/1000 | Direct synchronization | ||
| LSI | PLL Loop control (not recommended) | ||
| Slave | Any | None | |
| Slave | Master | Any | PLL Loop control |
| Slave | Any | None |
2.2.2. Implementation
Drift measurement can be done through the link layer that provides ISO anchor point timestamps through an HCI vendor specific event. When being slave, it contains information of the drift between the two sleeps clocks.
Then a mechanism using is used to compute drift between the local sleep clock and the local audio clock, using an approximately 1MHz free running timer, clocked by the PLL output (same clock as audio peripheral).
Finally, adjustment of the frequency can be done on the fly by updating the fractional N value of the PLL.
2.3. Audio latency
The BLE audio provides way on mastering the audio latency from the source to the sink. The total audio latency is the sum of the following sub latencies :
- Buffering delay is the time needed to buffers a frame and is mostly linked to the used codec.
- Algorithmic delay is intrinsic to the codec for ensuring signal continuity during mathematic transformations.
- Presentation delay may be negotiated by profiles within a provided supported range to ensure synchronization of several devices. It can be spitted into two sub delays
- Application delay in outside the controller, its minimum is related to the audio processing speed, ADC converter speed, etc ... while its maximum is linked to the resources allocated for buffering.
- Controller delay is inside the controller, its minimum is related to the codec processing speed and radio preparation time, while its maximum is linked to the resources allocated for buffering.
- Transport latency is introduced by the link layer and correspond to the maximum time for transmitting a packet over the isochronous link. An higher transport latency means more possibilities of retransmitting packet and an higher quality of service. While the maximum value is given by the profile to the link layer, the final latency is chosen by the link layer and can be computed from the ISO stream parameters (RTN, PHY, BN...).
As introduced, the Presentation delay is negotiated by the BLE audio profiles and useful for synchronizing when several Servers or Broadcast sink are involved. In Unicast, this delay -at Server side- is decided by the Client using ranges provided by all Servers, while in Broadcast this value is imposed to the sink. On the Client side, we don't have synchronization constraint and only speak about a Processing delay.
Locally, the device has to split this delay between the Controller delay and the Application delay. This decision in done at the application level based on the range of both delays. The range of controller delay can be read using a standard HCI command, but please note that the application is in charge of re-computing this value if several data path are superposed.
Creation of the ISO stream is done prior the establishment of the data path. That means the anchor point of the ISO event is the starting point of all timings and upper layers has to start audio based on that. Timestamps of the anchor point or SDU reference timings are provided by the link layer. A callback mechanism is provided by the codec manager to trig the audio peripheral -either at source or sink- for ensuring the Controller delay. Then the application is in charge of ensuring the Application delay if this latency is not null.
| Use case | Buffering delay | Application delay | Controller delay | Transport latency | Controller delay | Application delay | Algorithmic delay | Total delay |
|---|---|---|---|---|---|---|---|---|
| Phone Call, bidirectionnal at 24kHz, 10ms frame | 10ms | ~0ms | 4ms | 2.55ms | 4ms | 0.1ms | 2.5ms | 23.15ms |
| PBP Stereo at 48kHz | 10ms | 0.1ms | 22ms | 13.312ms | 39.9m | 0.1ms | 2.5ms | 87.9ms |
2.4. Codec Manager library
2.4.1. Overview
The Codec Manager is provided as a library and can be seen as a addon the the Controller. When used, this bloc must be linked to the controller, and must be integrated with local resources.
It provides a vendor specific data path on ID 1 that is a RAM with a flexible organization defined by sample depth and decimation parameters. These two parameters must be aligned when configuring the data path through the HCI Configure Data Path command that is using these vendor specific parameters.
Buffers numbers should be at least 2. For example, at audio source, when the first one is used for the DMA transfer, the second one (or more) can be given to the codec manager for encoding process, that must runs before before being erased again by the DMA.
2.4.2. Dependencies
The codec manager relies on the utility stm_queue.c.
The LC3 codec uses some mathematical functions of math.h and the Floating Point Unit.
2.4.3. Controller interface
BLE_Codec.c file is used for interaction between Controller and Codec Manager. Here, every HCI commands related the codec or the data path must be linked.
2.4.3.1. HCI commands
- Codec related commands
- HCI Read Local Supported Codecs
- HCI Read Local Supported Codec Capabilities
- HCI Read Local Supported Controller Delay
- Data path configuration related command
- HCI configure Data Path
- Data path commands
- HCI LE Setup ISO Data Path
- HCI LE Remove ISO Data Path
This last category of commands must not only be handled by the Codec Manager, but also by the link layer. The controller is in charge of splitting these commands depending on the parameters used.
2.4.3.2. Shunted events
Also the controller shunts some HCI commands and events related to the ISO Stream by calling the BLE_IsochronousGroupEvent() function with some parameters. Effectively, the Codec Manager expects to be notified of the creation and deletion of the ISO streams using the AUDIO_RegisterGroup() and AUDIO_UnregisterGroup() functions.
2.4.3.3. ISO data
- ISO data are also exchanged here. Data coming from the link layer are provided from the low ISR context, and must be provided to the codec using CODEC_ReceiveMediaPacket(). On the other direction, the codec will call BLE_SendIsoDataToLinkLayer()
2.4.3.4. Vendor specific synchronization mechanism
And finally, a vendor specific mechanism for synchronization and timings management has to be linked
- HCI vendor specific "synchronization event", containing timestamps related to the ISO group
- The calibration Callback - called from the low ISR context - necessary for matching timestamp from the link layer to the codec clock domain.
2.4.4. Application interface
The application is in charge of initializing the Codec Manager using the CODEC_ManagerInit() function. This function provides the codec buffers of RAM useful for the LC3 codec inside a structure of type CODEC_LC3Config_t, also the media_packet_pool is provided here. This pool is necessary for buffering media packets and providing a range of supported controller delay. Finally we provide here some timings margin to the controller delay, related to either the application or the link layer behavior (radio preparation time). Internal memories and states can be cleaned using the CODEC_ManagerReset().
Function CODEC_RegisterTriggerClbk() allows to register a callback in each direction for starting the audio peripheral (or DMA) at the rate of the frame duration.
Effectively, CODEC_SendData() and CODEC_ReceiveData() function must then be called every frame duration from the DMA ISR context.
- CODEC_SendData() notify the Codec Manager that new data is ready to be encoded
- CODEC_ReceiveData() notify the Codec Manager that a buffer has been played on the peripheral and is ready to be refilled by audio data
Both of these API may generate a processing request CODEC_ProcessReq() for asking to run the heavy codec processing by calling CODEC_ManagerProcess().
The internal clock corrector must be initialized, after the PLL has been setup, using function AUDIO_InitializeClockCorrector() and providing the details of the clock tree, and intervals of calculations. AUDIO_DeinitializeClockCorrector() can be used for resetting the clock corrector.
2.4.5. codec integration
Codec_if.c file is used for allowing the codec manager to access some resources.
2.4.5.1. Processing
CODEC_ProcessInit() and CODEC_ProcessReq(), as seen before, are related to codec processing. Since the codec manager has real time constraint but still is an heavy CPU load, it is recommended to either execute the codec manager from a interrupt with low priority or a task with high priority.
CODEC_ReqIRQState() function is here for delaying this processing and avoiding the codec to preempt another task. If the codec is delayed, the extra latency must be added to the minimum controller delay as a margin.
2.4.5.2. Audio Timer
The codec manager must relies on a approximately 1MHz timer. Initialization of this timer can be done inside CODEC_CLK_Init() that must be called from the application before the ISO stream is established. The clocking tree of this timer is critical since it is also use for clock synchronization, it absolutely needs to be clocked from the same clock as the audio peripheral, meaning from the PLL output. A dedicated 20 bits audio timer is available in the RCC IP but a general purpose timer could also be used.
Codec manager will call CODEC_CLK_GetHostTimestamp() for getting some timestamp from this clock.
This timer must be free running with the capability of requesting event on some given timestamps through the CODEC_CLK_RequestTimerEvent() API, then event is then notified with CODEC_CLK_trigger_event_notify() to the codec manager
2.4.5.3. PLL interaction
The PLL corrector needs CODEC_CLK_GetPLLNfrac() and CODEC_CLK_SetPLLNfrac() to be implemented for synchronizing clocks.
2.4.5.4. Debug signal and traces
Depending on the compilation option, some events or logs can be retrieved in CODEC_TraceEvnt() and CODEC_DBG_Log() functions.
2.4.6. Considerations
Looking closer at the link layer behavior, we notice it needs time for preparing an event. This additional timing is called radio setup time and any packet given to the link layer inside this preparation windows cannot be send immediately and will be delayed of one ISO event.
Thus, this delay is added to the minimum controller delay value based on the given parameters at the codec initialization. This value has to be profiled with the core clock used during audio streaming. As soon as the ISO stream is up, this value can also be retrieved from the aci_hal_sync_event() as the delta between (Next_Anchor_Point - Next_Sdu_Delivery_Timeout).
Considering this, and since the time when codec manager send its packets to the link layer is based on the controller delay, it is not recommended to use value of controller delay at source that may leads to get close to the identified boundary. At the application level, that means to avoid (controller_delay % frame_ms) ≈ radio setup time