encapp is a tool to test video encoders in Android.
It provides an easy way to test an android video encoder by easily combining parameters like:
- codecs
- bitrate
- framerate
- i-frame interval
- coding mode
- others
encapp also has support for dynamically changing framerate, bitrate, and ltr.
This document describes how to use the tool.
- for tool development, check README.dev.md.
- for details on test configuration, check README.test.md.
For running encapp:
- adb connection to the device being tested.
- ffmpeg with decoding support for the codecs to be tested
- install some python packages
- protobuf (https://developers.google.com/protocol-buffers/docs/downloads)
List of required python packages:
- humanfriendly
- numpy
- pandas
- seaborn
- protobuf (google protocol buffers)
(1) install the app and give it permission to access the external storage:
$ ./scripts/encapp.py install
$ adb shell appops set --uid com.facebook.encapp MANAGE_EXTERNAL_STORAGE allow
(2) run the list command:
$ ./scripts/encapp.py list
...
--- List of supported encoders ---
MediaCodec {
name: OMX.google.h264.encoder
type {
mime_type: video/avc
max_supported_instances: 32
color {
format: 2135033992
name: COLOR_FormatYUV420Flexible
}
color {
format: 19
name: COLOR_FormatYUV420Planar
}
...
}
}
*******
--- List of supported decoders ---
MediaCodec {
name: OMX.google.h264.decoder
type {
mime_type: video/avc
max_supported_instances: 32
color {
format: 2135033992
name: COLOR_FormatYUV420Flexible
}
...
}
}
Note: The scripts/encapp.py
scripts will install a prebuild apk before running the test. If you have several devices attached to your host, you can use either the "ANDROID_SERIAL
" environment variable or the "--serial <serial>
" CLI option.
First, select one of the codecs from step 4. In this case, we will use OMX.google.h264.encoder
.
Push the (raw) video file to be encoded into the device. Note that we are using a QCIF video (176x144). In this case the raw format is yuv420p (which is stated in the codec list from (2.2), COLOR_FormatYUV420Planar).0 For sw codecs in most case it is yuv420p. For hw codecs e.g. Qcom: COLOR_QCOM_FormatYUV420SemiPlanar this is nv12. In the case of surface encoder from raw the source should be nv21 regardless of codec.
$ wget https://media.xiph.org/video/derf/y4m/akiyo_qcif.y4m -O /tmp/akiyo_qcif.y4m
$ ffmpeg -i /tmp/akiyo_qcif.y4m -f rawvideo -pix_fmt yuv420p /tmp/akiyo_qcif.yuv
Now run the h264 encoder (OMX.google.h264.encoder
):
$ encapp.py run tests/bitrate_buffer.pbtxt
...
results collect: ['PATH/bitrate_files/encapp_XXX.json']
Results are copied into a directory called encapp_*
. They include:
- encoded video, using the mp4 container for h264, and the ivf container for vp8 and vp9
- json file containing per-frame information
Now, let's run the h264 encoder in an HD file. We will just select the codec ("h264"), and let encapp choose the actual encoder.
$ wget https://media.xiph.org/video/derf/y4m/KristenAndSara_1280x720_60.y4m -O /tmp/KristenAndSara_1280x720_60.y4m
$ ffmpeg -i /tmp/KristenAndSara_1280x720_60.y4m -f rawvideo -pix_fmt yuv420p /tmp/KristenAndSara_1280x720_60.yuv
$ encapp.py run tests/bitrate_buffer.720p.pbtxt
...
results collect: ['PATH/bitrate_buffer.720p_files/encapp_XXX.json'
First at all create your own proto configuration file: Get the codec from the codec list.
test {
common {
id: "X"
description: "My test is simple"
}
input {
filepath: "mediaX.yuv"
resolution: "WxH"
framerate: X
}
configure {
codec: "X"
bitrate: "XXX kbps"
}
}
Definitions of the keys in the proto buf definition: proto/tests.proto
Additional settings (besides bitrate etc).
Example:
configure {
codec: "encoder.avc"
bitrate: "100 kbps"
bitrate_mode: cbr
i_frame_interval: 2000
parameter {
key: "vendor.qti-ext-enc-ltr-count.num-ltr-frames"
type: intType
value: "3"
}
parameter {
key: "vendor.qti-ext-enc-caps-ltr.max-frames"
type: intType
value: "3"
}
}
Each setting consists of a pair {FRAME_NUM, VALUE}
, where the VALUE can be an empty string.
- Dynamically change framerate:
runtime {
video_bitrate {
framenum: 60
bitrate: "50k"
}
video_bitrate {
framenum: 120
bitrate: "100k"
}
video_bitrate {
framenum: 180
bitrate: "400k"
}
}
- Low latency (Android API 30)
decoder_runtime
{
parameter {
framenum: 60
key: low-latency"
type: intType
value: 1
}
}
The names json result files does not give any clues as to what settings have been used. To figure that out run:
$ encapp_search.py
Running it without any arguments will index all parsable json files in the current folder and below.
To find all 720p files run:
$ encapp_search.py -s 1280x720
Example: Calculate quality degradation
Run
$ encapp_quality.py --header --media MEDIA_FOLDER $(encapp_search.py)
Since the json file only contains the name of the source for an encoding the source folder needs to be provided. The output will be a csv file (default name is 'quality.csv') containing vmaf, ssim, psnr and other relevant properties.
Just install the pip packages.
$ sudo pip install humanfriendly numpy pandas seaborn protobuf
Make sure your pip3 and your default python3 shell are coherent.
bash-3.2$ sudo chown -R $(whoami) /usr/local/bin /usr/local/lib /usr/local/sbin
bash-3.2$ brew install python@3.9
...
bash-3.2$ brew link --overwrite python@3.9
Linking /usr/local/Cellar/python@3.9/3.9.12... 24 symlinks created.
bash-3.2$ pip3 install protobuf
...
Collecting protobuf
Downloading protobuf-3.19.4-cp39-cp39-macosx_10_9_x86_64.whl (961 kB)
...