Notes
Process
CAE is enabled on a Zencoder Job by setting "type": "cae" on a Zencoder output. In addition, cae_options may also be configured.
Zencoder "type": "cae" output runs the context aware profile generator on the given input and the resulting profile generated as a json file is uploaded to the user provided location.
The generated profile json contains following settings that are used for encoding each rendition in the ladder.
widthheightframe_ratevideo_codecvideo_codec_profilevideo_codec_tiervideo_codec_levelvideo_reference_framesvideo_bframesvideo_bitratedecoder_bitrate_capdecoder_buffer_sizekeyframe_ratefixed_keyframe_interval
For Zencoder to automatically encode the renditions, additional dependent outputs are required and each output with cae_rendition set. Other Outputs without a cae_rendition will begin encoding as soon as the Input source is inspected, concurrently while the dynamic profile generator is running.
Sample request
{
"input": "s3://test_bucket/test_media.mp4",
"outputs": [
{
"type": "cae",
"label": "cae_profile",
"url": "s3://test_bucket/cae_generated_profile.json",
"cae_options": {
"min_renditions": 1,
"max_renditions": 6
}
},
{"source": "cae_profile", "cae_rendition": 1}, {"source": "cae_profile", "cae_rendition": 2},
{"source": "cae_profile", "cae_rendition": 3}, {"source": "cae_profile", "cae_rendition": 4},
{"source": "cae_profile", "cae_rendition": 5}, {"source": "cae_profile", "cae_rendition": 6}
]
}Guidelines for configuring cae_options
Number of Renditions
The minimum and maximum number of renditions to create is controlled with the min_renditions and max_renditions settings. For best performance, it is recommended to leave a certain gap between these limits (e.g. set min_renditions to 2 and max_renditions to 10), allowing the CAE profile generator to select the number most suitable for each content. For easy to encode content, it may produce fewer renditions, while for more complex content it may produce more. It is also recommended to set min_renditions to be as small as possible, to allow for saving bandwidth when more renditions are unnecessary.
Resolutions
By default, CAE uses a ladder of about 30 standard resolutions, ranging from 192x108 to 7680x4320. It can be limited on either the low or high end by using the min_resolution and max_resolution settings. Alternately, a custom list of resolutions and associated codec settings can be provided in the video_configurations setting.
GOP length
The keyframe_rate setting controls the maximum GOP length of the encoded content. Set this to correspond with segment_seconds for HLS/DASH encoding. Otherwise a value of 0.5 is recommended, corresponding to a 2-second GOP length.
Bitrate coverage
The limits of the bitrates to be used can controlled by the min_bitrate and max_bitrate settings. This will instruct CAE to allocate at least min_bitrate Kbps for the first rendition and constrain the last rendition to at most max_bitrate Kbps.
To ensure the rendition bitrates are spaced appropriately, use the min_granularity and max_granularity settings. For example, these can be used to make sure the step from one bitrate up to the next is not less than a 50% increase in bitrate or more than a 100% increase in bitrate. Please note that most streaming authoring guidelines recommend granularity doesn’t exceed 100%.
Startup latency, reliability at poor network conditions
To ensure the bitrate ladder always starts with a low enough bitrate, use the max_first_rendition_bitrate setting. The lower the first rendition bitrate is set, the less likely a video player will need to buffer due to low connection speeds. Additionally, if the streaming manifest indicates to load the lowest bitrate first, this will also directly affect startup latency.
Support for legacy devices
To support older devices that are only capable of Baselineselect_baseline_profile_configuration setting. When this is set, at least one rendition will be encoded with Baseline profile. Additional constraints (codec profile, level, number of reference frames, b-frames, etc.) can be controlled with a custom video_configurations setting.
Note that select_baseline_profile_configuration is true by default, and it requires at least one rendition to have a resolution at or below 512x288.
VBR variability control
The maximum peak bitrates can be controlled with the bitrate_cap_to_bitrate_ratio and bitrate_cap_offset settings. If a bitrate “R” is selected for a rendition, then the decoder_bitrate_cap is calculated as:
decoder_bitrate_cap = R * bitrate_cap_to_bitrate_ratio + bitrate_cap_offset
Similarly, the decoder buffer limits can be controlled with the buffer_size_to_bitrate_ratio and buffer_size_offset settings. The decoder_buffer_size is calculated as:
decoder_buffer_size = R * buffer_size_to_bitrate_ratio + buffer_size_offset
Note that peak bitrate and buffer limits may also be constrained by the codec profile and/or level settings. Some general device compatibility issues can be avoided by limiting bitrate_cap_to_bitrate_ratio to 2.0 or below, and buffer_size_to_bitrate_ratio to 2.0 * GOP length (in seconds).
CAE and segmented outputs
CAE is compatible with segmented outputs in Zencoder. On the Output rendition settings, set the type as segmented and the appropriate segment_seconds value. Also, keyframe_rate and fixed_keyframe_interval settings must be set in cae_options.
CAE - HLS example
{
"input": "s3://test_bucket/test_media.mp4",
"outputs": [
{
"type": "cae",
"label": "cae_profile",
"base_url": "s3://test_bucket/outputs/hls_cae/",
"filename": "cae_options.json",
"cae_options": {
"max_renditions": 6,
"min_renditions": 2,
"keyframe_rate": "0.5",
"fixed_keyframe_interval": "true"
}
},
{
"source": "cae_options",
"label": "cae_1",
"cae_rendition": 1,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_1",
"filename": "cae_1.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"source": "cae_options",
"label": "cae_2",
"cae_rendition": 2,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_2",
"filename": "cae_2.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"source": "cae_options",
"label": "cae_3",
"cae_rendition": 3,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_3",
"filename": "cae_3.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"source": "cae_options",
"label": "cae_4",
"cae_rendition": 4,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_4",
"filename": "cae_4.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"source": "cae_options",
"label": "cae_5",
"cae_rendition": 5,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_5",
"filename": "cae_5.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"source": "cae_options",
"label": "cae_6",
"cae_rendition": 6,
"base_url": "s3://test_bucket/outputs/hls_cae/cae_6",
"filename": "cae_6.m3u8",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"generate_keyframe_manifest": "true",
"hls_protocol_version": 7,
"skip_audio": true
},
{
"label": "Audio",
"audio_bitrate": "128",
"type": "segmented",
"segment_seconds": 2,
"byte_range_segmenting": true,
"skip_video": true,
"base_url": "s3://test_bucket/outputs/hls_cae/Audio/128k/",
"filename": "audio_128k.ac3"
},
{
"base_url": "s3://test_bucket/outputs/hls_cae/",
"filename": "master_playlist.m3u8",
"hls_protocol_version": 7,
"type": "playlist",
"allow_skipped_sources": true,
"alternate_audio": {
"128k_audio": {
"path": "Audio/128k/audio_128k.m3u8",
"language": "en",
"source": "Audio"
}
},
"streams": [
{
"path": "cae_1/cae_1.m3u8",
"source": "cae_1",
"audio": "128k_audio"
},
{
"path": "cae_2/cae_2.m3u8",
"source": "cae_2",
"audio": "128k_audio"
},
{
"path": "cae_3/cae_3.m3u8",
"source": "cae_3",
"audio": "128k_audio"
},
{
"path": "cae_4/cae_4.m3u8",
"source": "cae_4",
"audio": "128k_audio"
},
{
"path": "cae_5/cae_5.m3u8",
"source": "cae_5",
"audio": "128k_audio"
},
{
"path": "cae_6/cae_6.m3u8",
"source": "cae_6",
"audio": "128k_audio"
}
]
}
]
}
CAE - DASH example
{
"input": "s3://test_bucket/test.mov",
"outputs": [
{
"type": "cae",
"label": "cae_profile",
"base_url": "s3://test_bucket/outputs/hls_cae/",
"filename": "cae_options.json",
"cae_options": {
"max_renditions": 6,
"min_renditions": 2,
"keyframe_rate": "0.5",
"fixed_keyframe_interval": "true"
}
},
{
"source": "cae_options",
"label": "cae_1",
"cae_rendition": 1,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_1",
"filename": "cae_1.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"source": "cae_options",
"label": "cae_2",
"cae_rendition": 2,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_2",
"filename": "cae_2.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"source": "cae_options",
"label": "cae_3",
"cae_rendition": 3,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_3",
"filename": "cae_3.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"source": "cae_options",
"label": "cae_4",
"cae_rendition": 4,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_4",
"filename": "cae_4.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"source": "cae_options",
"label": "cae_5",
"cae_rendition": 5,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_5",
"filename": "cae_5.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"source": "cae_options",
"label": "cae_6",
"cae_rendition": 6,
"base_url": "s3://test_bucket/outputs/dash_cae/cae_6",
"filename": "cae_6.mpd",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_audio": true,
"public": true
},
{
"label": "Audio",
"audio_bitrate": "128",
"type": "segmented",
"segment_seconds": 10,
"streaming_delivery_format": "dash",
"skip_video": true,
"base_url": "s3://test_bucket/outputs/dash_cae/Audio/128k/",
"filename": "audio_128k.mpd",
"public": true
},
{
"base_url": "s3://test_bucket/outputs/dash_cae/",
"filename": "master_playlist.mpd",
"streaming_delivery_format": "dash",
"type": "playlist",
"allow_skipped_sources": true,
"public": true,
"streams": [
{
"path": "cae_1/cae_1.mpd",
"source": "cae_1"
},
{
"path": "cae_2/cae_2.mpd",
"source": "cae_2"
},
{
"path": "cae_3/cae_3.mpd",
"source": "cae_3"
},
{
"path": "cae_4/cae_4.mpd",
"source": "cae_4"
},
{
"path": "cae_5/cae_5.mpd",
"source": "cae_5"
},
{
"path": "cae_6/cae_6.mpd",
"source": "cae_6"
},
{
"path": "Audio/128k/audio_128k.mpd",
"source": "Audio"
}
]
}
]
}