Skip to content

Debugging the Observation Framework

Jump to bottom
rcottingham edited this page Dec 12, 2024 · 1 revision

This section provide information about debugging the Device Observation Framework.

The Device Observation Framework must be run with access to the Test Runner so that it can send the observation results to it. To help debugging there is an option to run the Observation Framework in debugging mode without access to the Test Runner; in this case the observation results will be saved in a local folder.

NOTE

Running the Observation Framework in debugging mode should be used carefully and only for debugging purpose.

Preparing the shared configuration

There are two shared configuration files between Test Runner and Observation Framework, and they are both defined in dpctf-tests.

  • test-config.json: defines shared parameters such as tolerance
  • tests.json: defines test code with content details
  1. Create a "configuration" folder on Observation Framework root directory e.g: "device-observation-framework”.
  2. Download "test-config.json" and "tests.json" files from dpctf-tests repository and save it inside the "configuration" folder.

NOTE

It might cause the Observation Framework results to be different to the original results due to incorrect and/or different set up of the shared configuration files. It might cause exception and unable to process with Observation Framework at all if you choose to run with old recordings taken on previous version of Test Runner.

Run Observation Framework in debugging mode

Run Observation Framework in debugging mode and enables the debug log.

python observation_framework.py --input <recording_file> --mode debug --log debug

Observation Framework results and logs

When running Observation Framework in debugging mode the results will be saved under local directory in "results" folder and debugging logs will be saved under "logs" folder. "qr_code_list.csv" contains full detected QR code information which is useful for debugging.

The following information can be found from the QR code log csv file:

  • Camera Frame: Camera frame number from the recording where the QR code is detected.
  • Content ID, Media Time, Frame Number, Frame Rate: Information for presented video frame, the information is embedded on video Mezzanine.
  • Test Status, Last Action, Current Time, Delay: Test status from Test Runner and HTML current time with delay in generating QR code. The delay is reported on next status QR code.
  • Location: QR code position related to the captured screen.
  • Session ID, Test ID: Unique hex code of session ID and test ID which are detected from pre-test QR code. This information is from test runner.

Observation algorithms are documented here https://github.com/cta-wave/device-observation-framework/wiki/Observation-Algorithms to help debugging. For any missing QR code detection user will need to check from the recording file.

device-observation-framework

Clone this wiki locally