Skip to content

Libavcodec Compression

Jump to bottom
Martin Pulec edited this page Jul 10, 2024 · 30 revisions

FFmpeg provides a huge collection of codecs for use. Some of them (but not all) may be used with UltraGrid.

Table of Contents

Basics

When no compression is specified, MJPEG is used. You can, however, pick another compression like H.264 or HEVC. Selection of other codecs is available as well (i. a. ProRes, VP9 or AV1).

MJPEG

Basic, but still useful and computationally decent compression is MJPEG which can be used if invoked in a following way:

uv -t testcard -c libavcodec:codec=MJPEG
uv -t testcard -c libavcodec:codec=MJPEG:q=23         # use quality 23 (approximately default)
uv -t testcard -c libavcodec                          # MJPEG is a default compression
uv -t testcard -c libavcodec:encoder=mjpeg_qsv:cqp=80 # QSV mjpeg uses normal JPEG range [0..100]

Note: quality can be affected only by the "q" parameter. Other quality-affecting parameters like "bitrate" are not honored.

H.264

In contrary to HEVC and AV1, H.264 supports interlaced video so it could be the preferred choice for interlaced content.

Usage:

uv -t deltacast -c libavcodec:codec=H.264 <address>                 # use H.264
uv -t deltacast -c libavcodec:codec=H.264:bitrate=20M <address>     # specifies requested bitrate
uv -t deltacast -c libavcodec:codec=H.264:subsampling=420 <address> # enforce 4:2:0 subsampling (may be needed for the stream to be decodable with VAAPI)

Using NVENC encoding (NVIDIA only):

uv -t deltacast -c libavcodec:encoder=h264_nvenc

Use CUVID (HW accelerated) decoder (see below):

uv -d gl --param force-lavd-decoder=h264_cuvid

HEVC

Since HEVC is relatively new and still a bit demanding compression providing a great compression ratio, you may need to tweak things a bit to achieve optimal performance.

There are multiple encoders supporting HEVC encoding, namely libx265 and hevc_nvenc (and also hevc_qsv if available). While encoding HEVC is still a bit demanding, it is advisable to use the NVENC encoder (or QSV) to encode the stream:

uv -t deltacast -c libavcodec:encoder=libx265 <addr>   # x265 is slower to encode but parallelizable on decoder
uv -t deltacast -c libavcodec:encoder=hevc_nvenc <addr>
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:bitrate=15M <addr> # reduce bitrate to be easier to decode
uv -t deltacast -c libavcodec:encoder=hevc_nvenc:spatial_aq=1  # improves quality at the expense of higher
                                                               # complexity, HW decoder may be required
uv -t deltacast -c libavcodec:encoder=hevc_qsv:cqp=5000        # use QSV with CQP=5000 (usable range approx. 2000-8000)

Currently, the stream encoded by NVENC encoder isn’t much parallelizable by decoder, so you may want to force hardware decoder (please note that the decoder currently adds some 4 frames of latency!):

uv -d gl --param force-lavd-decoder=hevc_cuvid

AV1

AV1 is supposed to be more efficient than HEVC but also more performance demanding, mainly on the encoder.

Decoding

Nothing special is required for the decompression, libdav1d is the preferred decoder.

Encoding

There are multiple encoders - rav1e, libaom-av1 and libsvtav1. The later 2 are recommended since rav1e currently (2024) lags behind those 2 in terms of performance. libsvtav1 is usually faster but there are some exceptions, like ARM-based Macs.

libsvtav1 supports multiple rate-control modes with different properties: VBR is triggered by passing bitrate option without cbr/cqp; not recommended, increases latency, see below), CBR (cannot be selected by now, see also below), CRF (implicit recommended) and CQP (also OK). For both CRF and CQP, the bitrate can be specified which sets MaxBitRate.

Examples:

  • SVT AV1: uv -t <cap> -c lavc:enc=libsvtav1[:crf=35]

    Use CRF as the rate control mode (recommended).

    CRF=35 is implicit/default, range 0-63, the smaller value the better compression (higher bitrate)

  • SVT AV1 with CRF and bitrate: uv -t <cap> -c lavc:enc=libsvtav1:crf=35:bitrate=3M

    also uses CRF=35 with bitrate limited to 3 Mbps

  • (SVT AV1 with bitrate: uv -t <cap> -c lavc:enc=libsvtav1:bitrate=3M)

    use VBR with 3 Mbps. Currently not recommended because (with current options) greatly increases latency. CBR is unfortunately not possible to set by now due to incorrect settings in FFmpeg.

    setting cpb_props->max_bitrate to a non-zero is refused by SVT AV1 in CBR mode. max_bitrate is must be set to >0 otherwise the FFmpeg plugin doesn't set CBR (see the link few lines above)

  • SVT AV1 with QP: uv -t <cap> -c lavc:enc=libsvtav1:cqp=30

  • AOM AV1: uv -t <cap> -c lavc:enc=libaom-av1:bitrate=3M

Other compressions

Besides H.264/HEVC, UltraGrid supports also few other codecs that may be used, eg.:

  • VP8/VP9
  • J2K - FFMPEG's implementation of J2K is however very slow
uv -t deltacast -c libavcodec <addr>             # use default libavcodec codec (currently MJPEG)
uv -t deltacast -c libavcodec:codec=MJPEG <addr> # use MJPEG codec explicitly
uv -t deltacast -c libavcodec:help <addr>        # prints available codecs/encoders (and decoders)

HW accelerations

See also a separate page Hw. acceleration support.

UltraGrid offers some hardware compression accelerations. Encoders are always standalone, therefore specifying encoder=<name> is sufficient. For decoders there are on one hand also standalone decoders that may be used similarly but also so called hardware accelerations (currently implemented in UG only for Linux).

Please note that the HW decoders may not be able to decode arbitrary video. If decoder refuses to decode, it is useful to try to set encoder to encode 8-bit 4:2:0 YCbCr video. You can try -c libavcodec[other_opts]:subsampling=420, if it doesn't help. :disable_intra_refresh may also improve compatibility. Color depth should be ideally controlled at the source (eg. -t decklink:codec=UYVY).

Encode

HW accelerated encoding is toggled by selecting appropriate encoder, eg. hevc_vaapi or hevc_nvenc:

uv -t testcard -c libavcodec:encoder=hevc_amf (Windows only)
uv -t testcard -c libavcodec:encoder=hevc_nvenc
uv -t testcard -c libavcodec:encoder=hevc_vaapi (Linux only)
uv -t testcard -c libavcodec:encoder=hevc_qsv
uv -t testcard -c libavcodec:encoder=hevc_videotoolbox (mac only)

Decode

VA-API and VDPAU (Linux)

VAAPI/VDPAU accelerated decoding (if supported) can be toggled with following command:

uv -d decklink --param use-hw-accel

CUVID, QuickSync

CUVID and QuickSync are regular, HW-accelerated decoders (can be listed with -c libavcodec:help). Usage:

uv -d gl --param force-lavd-decoder=hevc_cuvid
uv -d gl --param force-lavd-decoder=hevc_qsv

or if unsure which codec arrives, you can list more decoders:

uv -d gl --param force-lavd-decoder=hevc_cuvid:h264_cuvid

Note: Using a cuvid may introduce additional latency.

Note 2: QuickSync decoder may not support intra refresh feature, in that case use -c libavcodec:encoder=<enc>:disable_intra_refresh on encoder.

Prefer CUVID/QuickSync and similar decoders for higher bit depths because current UltraGrid VAAPI/VDPAU implementation supports only 8-bits decoding.

Note: If using VAAPI for decoding, set 4:2:0 subsampling on the encoder, otherwise decoding won't work, eg.:

uv -t testcard -c libavcodec:encoder=libx264:subsampling=420 <receiver>

Compatibility table

API Encode Decode
NVENC Win/Linux Win/Linux (CUVID)
QuickSync Win/Linux ¹ Win/Linux ¹
VideoToolbox Mac Mac
VA API ² Linux Linux
VDPAU ² Linux Linux

See also:

Advanced

Fine-tuning stream parameters

UltraGrid parameters allow more fine-grained selecting of parameters, eg. converting R12L to 10 or 12-bit 4:4:4 YCbCr, options are:

uv -t testcard -c libavcodec:encoder=libx265[:rgb|:yuv][:depth=<d>][:subsampling=<s>]O

Or you may still leave the selection on UltraGrid but changing the precedence policy. You may prefer keeping color-space instead of bit-depth and subsampling (whichi is default). This may be useful in particular to preserve performance preventing doing potentially costly color-space transformations:

uv -t testcard -c libavcodec:encoder=libx265 --conv-policy cds # == [c]olor>[d]epth>[s]ubsampling
uv -d deckink --conv-policy cds                                # works on receiver as well

There are also original no longer recommended options lavc-use-codec at encoder and decoder-use-codec:

uv -t decklink:codec=R12L --param lavc-use-codec=yuv444p16le -c libavcodec:encoder=hevc_nvenc <recv>
uv -d decklink --param decoder-use-codec=R12L <sender>

Here the codec is one that is supported by the display. If you enter an unsupported one, decoder writes you available options. Or you can use a keyword help to see display supported codecs.

Checking available pixel formats

It is sometimes tricky to know which pixel formats are available and provided by the decoder. Fortunately it can be probed with UltraGrid, just run (for example for x264; don't forgot to press Ctrl-C to exit UltraGrid):

uv -t testcard -c libavcodec:encoder=libx264 --verbose=7 2>&1 | grep 'supported pixel formats'

You can also check what codec would go from decoder. This is only informative, UltraGrid can do a conversion if needed. But if you expect 12-bit and there are only 8-bits available, it would need more tweaking (another decoder or a card):

uv -t testcard -c libavcodec:encoder=libx264 -d dummy --verbose=7 2>&1 | grep 'Available output pixel formats:'

Note: the available output pixel formats may differ according to input stream properties, thus pass the parameters to encoder exactly.

Checking supported pixel formats

Sometimes is easier to get an overview of device capabilities first.

NVENC/NVDEC

To see capabilities of a graphic card, you can download and compile this tool (requires nv-codec-headers) and see output of nvencinfo and nvdecinfo for an list of HW supported features

git clone https://github.com/FFmpeg/nv-codec-headers.git
cd nv-codec-headers && make && sudo make install
cd ..
git clone https://github.com/philipl/nv-video-info
cd nv-video-info && ./autogen.sh && make
./nvencinfo

Also this matrix may be useful.

VDPAU/VAAPI

For VDPAU and VAAPI there exist functions vdpauinfo and vainfo to check capabilities

If you have any technical or non-technical question or suggestion please feel free to contact us at

[email protected]

Table of Contents

Clone this wiki locally