Android13 Camera New Feature - Camera timestamp base support

Starting from Android 13, Android supports set timestamp base for each Camera Output Stream. This article introduces how to use it.

OutputConfiguration new APIs for timestamp base

Constants part

Android 13 added the following constants in OutputConfiguration class to represent different timestamp base:

Constants	Value	Description
TIMESTAMP_BASE_CHOREOGRAPHER_SYNCED	4	Timestamp is synchronized to choreographer.
TIMESTAMP_BASE_DEFAULT	0	Default timestamp base.
TIMESTAMP_BASE_MONOTONIC	2	Timestamp base roughly the same as SystemClock.uptimeMillis().
TIMESTAMP_BASE_REALTIME	3	Timestamp base roughly the same as SystemClock.elapsedRealtime().
TIMESTAMP_BASE_SENSOR	1	Timestamp base of CameraCharacteristics#SENSOR_INFO_TIMESTAMP_SOURCE.

TIMESTAMP_BASE_CHOREOGRAPHER_SYNCED

The timestamp of the output images are overridden with choreographer pulses from the display subsystem for smoother display of camera frames.

• An output target of SurfaceView uses this time base by default.
• This timestamp base isn't applicable to SurfaceTexture targets. SurfaceTexture's updateTexImage function always uses the latest image from the camera stream. In the case of a TextureView, the image is displayed right away.
• Timestamps with this time base cannot directly match the timestamps in CameraCaptureSession.CaptureCallback#onCaptureStarted or the sensor timestamps in CaptureResult.
• This timestamp base shouldn't be used if the timestamp needs to be used for audio-video synchronization.

TIMESTAMP_BASE_DEFAULT

The camera device decides the timestamp based on the properties of the output surface.

• For a SurfaceView output surface, the timestamp base is TIMESTAMP_BASE_CHOREOGRAPHER_SYNCED. The timestamp is overridden with choreographer pulses from the display subsystem for smoother display of camera frames. The timestamp is roughly in the same time base as SystemClock.uptimeMillis().
• For an output surface of MediaRecorder, MediaCodec, or ImageReader with HardwareBuffer.USAGE_VIDEO_ENCODE usge flag, the timestamp base is TIMESTAMP_BASE_MONOTONIC, which is roughly the same time base as SystemClock.uptimeMillis().
• For all other cases, the timestamp base is TIMESTAMP_BASE_SENSOR, the same as what's specified by CameraCharacteristics#SENSOR_INFO_TIMESTAMP_SOURCE.

TIMESTAMP_BASE_MONOTONIC

The timestamps of the output images are monotonically increasing, and are roughly in the same time base as SystemClock.uptimeMillis(). The timestamps with this time base can be directly used for audio-video sync in video recording.

If the camera device's CameraCharacteristics#SENSOR_INFO_TIMESTAMP_SOURCE is REALTIME, timestamps with this time base cannot directly match the timestamps in CameraCaptureSession.CaptureCallback#onCaptureStarted or the sensor timestamps in CaptureResult.

TIMESTAMP_BASE_REALTIME

The timestamps of the output images are roughly in the same time base as SystemClock.elapsedRealtime(). The timestamps with this time base cannot be directly used for audio-video sync in video recording.

If the camera device's CameraCharacteristics#SENSOR_INFO_TIMESTAMP_SOURCE is UNKNOWN, timestamps with this time base cannot directly match the timestamps in CameraCaptureSession.CaptureCallback#onCaptureStarted or the sensor timestamps in CaptureResult.

If using a REALTIME timestamp base on a device that supports only TIMESTAMP_SOURCE_UNKNOWN, the accuracy of timestamps is only what is guaranteed in the documentation for UNKNOWN. In particular, they have no guarantees about being accurate enough to use in fusing image data with the output of inertial sensors, for features such as image stabilization or augmented reality.

TIMESTAMP_BASE_SENSOR

The timestamps of the output images are in the time base as specified by CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE. The application can look up the corresponding result metadata for a particular output image using this timestamp.

APIs part

Android 13 added two APIs in OutputConfiguration class to set and get the timestamp base of the current OutputConfiguration.

setTimestampBase

Syntax:

public void setTimestampBase (int timestampBase)

Description:

• Set timestamp base for this output target.
• Timestamp base describes the time domain of images from this camera output and its relationship with CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE.
• If this function is not called, the timestamp base for this output is TIMESTAMP_BASE_DEFAULT, with which the camera device adjusts timestamps based on the output target.

getTimestampBase

Syntax:

public int getTimestampBase()

Description:

• Get the current timestamp base.
• If no setTimestampBase(int) is called first, this function returns TIMESTAMP_BASE_DEFAULT.

setTimestampBase Examples

TIMESTAMP_BASE_DEFAULT

when we set preview OutputConfiguration's timestamp base to TIMESTAMP_BASE_DEFAULT, the timestamp from SurfaceTexture can match with the timestamp from onCaptureStarted:

TIMESTAMP_BASE_CHOREOGRAPHER_SYNCED

when we set preview OutputConfiguration's timestamp base to TIMESTAMP_BASE_CHOREOGRAPHER_SYNCED, we can see the timestamp from SurfaceTexture cannot match with the timestamp from onCaptureStarted.

2022-07-13 23:37:15.766 6382-6440/net.sourceforge.opencamera D/CameraController2: [TimeStamp] onCaptureStarted timestamp:517075956800
2022-07-13 23:37:15.766 6382-6382/net.sourceforge.opencamera D/Preview: [TimeStamp] SurfaceTexture timestamp:517144237253
2022-07-13 23:37:15.772 6382-6440/net.sourceforge.opencamera D/CameraController2: [TimeStamp] onCaptureStarted timestamp:517109885700
2022-07-13 23:37:15.802 6382-6440/net.sourceforge.opencamera D/CameraController2: [TimeStamp] onCaptureStarted timestamp:517146505800
2022-07-13 23:37:15.830 6382-6440/net.sourceforge.opencamera D/CameraController2: [TimeStamp] onCaptureStarted timestamp:517177273100
2022-07-13 23:37:15.831 6382-6382/net.sourceforge.opencamera D/Preview: [TimeStamp] SurfaceTexture timestamp:517244237249