1. Introduction
As seen in [Hardware handshake mode], encoding in 'Hardware Handshake' mode is interesting because it allows for reducing memory footprints. However, it presents some real-time constraints that need to be understood, especially if it involves integrating a new camera. We will detail these constraints using the timing of the IMX335 and OV5640 as examples.
2. Hardware handshake description
2.1. Blocks, signals, and interruptions
venc_rdy
This signal is used by DCMIPP to notify VENC when a set of lines (slice) is ready to be encoded. This signal is hidden from the CPU and is used when DCMIPP starts in hardware handshake mode.
venc_gbl_int/VENC_IRQ
This interrupt notifies the CPU of the main encoding events, such as the end of frame encoding or errors.
DCMIPP_IRQ
This interrupt notifies the CPU of the main acquisition events, such as the end of frame capture, end of line capture, or errors.
rstn_venc
Reset signal.
ck_icn_s_venc/ck_icn_m_venc
Input clocks.
2.2. Sequence diagrams
StartCapture()
The firmware requests DCMIPP to start capturing camera frames in hardware handshake mode. Once the capture starts, it continues at its own pace, regardless of encoding requests performed by the firmware.
venc_rdy
DCMIPP informs VENC when the specified number of lines has been received. This step is not visible to the firmware. If, for some reason, VENC frame encoding has not yet started, it misses some slices. For this reason, the firmware must check that the first image line has not yet been received before it starts VENC.
H264EncodeFrame() / EncodeConfig
The firmware requests VENC to start encoding one frame. It writes to the VENC register with the appropriate values, such as encoding parameters and buffer address.
EncodeSlice
The VENC hardware encodes the slice provided by the DCMIPP. The encoding process will really start on venc_rdy.
DCMIPP_IRQ(FRAME_CAPTURED)
DCMIPP signals to the firmware that a complete frame is captured.
VENC_IRQ(FRAME_READY)
VENC signals to the firmware that a complete frame is encoded.
2.3. Real time constraints
Once started by DCMIPP, the video capture continues at its own pace, regardless of what happens in the rest of the system.
DCMIPP sends the 'venc_rdy' signal to the encoder but does not receive any feedback in return.
The first timing constraint is that the firmware must start the frame encoding before the first 'venc_rdy' of the frame.
Otherwise, the image shifts, resulting in a characteristic image that scrolls from top to bottom.
The second timing constraint is that the encoder must have time to encode one slice between two 'venc_rdy' signals.
A potential problem is that the encoding time might vary depending on the complexity of the image.
If the encoding time does not fit within the reception interval of two lines, a timeout occurs, and an error (FUSE_ERROR) is reported to the firmware.
In this case, the frame is lost, and VENC must be reset. It is important to note that the frame capture must be stopped during this reset.
Undefined behavior may occur if VENC is reset while capture is running in parallel, including memory bus lock.
The fact that the capture might not be monotonic depending on the camera settings is indeed the sticking point of the 'hardware handshake' use case.
Among these settings, 'vertical blanking' is a key factor.
Vertical blanking is the interval between the last lines captured of frame n and the first lines captured of frame n + 1.
3. Examples
The examples below illustrate the real-time constraints that have just been mentioned in the use cases of the IMX335 and the OV5640.
Measured were:
- The acquisition timings of the slices: 'Line_Event_Callback' ~ venc_rdy
- The acquisition timings of the frames: 'Frame_Event_Callback' = DCMIPP_IRQ(FRAME_CAPTURED)
- The encoding timings of the frames: 'H264EncStrmEncode' = H264EncodeFrame() => VENC_IRQ(FRAME_READY)
3.1. IMX335 vs OV5640
3.1.1. IMX335/1080p30
3.1.2. OV5640/1080p30
3.1.3. Conclusion
The vertical blanking for the IMX355 is more significant.
This has two consequences: the first real-time constraint is easier to achieve (start encoding before capturing the first slice). However, the interval between each slice is smaller, making the second constraint more likely to be violated.
3.2. 3.1. IMX335 1080p30 vs 1080p15 [edit | edit source]
3.2.1. IMX335/1080p15
3.2.2. Conclusion
Though the frame period is 66 ms, the actual acquisition is completed within 28 ms, which is similar to 1080p30.
This brings almost the same real-time constraint on encoding as 1080p30 (peak memory bandwidth).