Transport stream audio ahead of test start lead to some audio observations to fail

[yanj-github]

While the test runner is presenting the pre-test QR code, the TS audio continues to play. This likely causes audio observations to fail at the beginning, as the OF is unable to detect audio at 0.0ms effectively and cause the audio sample presenting test to fail.
Additionally, this issue impacts the accuracy of the duration and start-up delay measurements.

Affected observations:

• [OF] When examined as a continuous sequence of timestamped audio samples of the audio stream, the 20ms test audio samples shall be a complete rendering of the source audio track and are rendered in increasing presentation time order.
• [OF] Audio start-up-delay: The start-up delay shall be sufficiently low.
• [OF] Audio: The playback duration shall match the duration of the CMAF Track.

[yanj-github]

• [OF] When examined as a continuous sequence of timestamped audio samples of the audio stream, the 20ms test audio samples shall be a complete rendering of the source audio track and are rendered in increasing presentation time order.

Not OF issue cta-wave/dpctf-tests#205 is raised.

• [OF] Audio start-up-delay: The start-up delay shall be sufficiently low.

This is OF issue the fixes will be committed soon.

• [OF] Audio: The playback duration shall match the duration of the CMAF Track.

Some of these are OF issue the fixes will be committed soon.

[jpiesing]

@yanj-github As far as reasonable, please could you give a little more information than above about which issues you do not believe are OF bugs & why. IMHO that will help guide what comes next.

[yanj-github]

@jpiesing in case you missed it. I’ve already created an issue cta-wave/dpctf-tests#205 with some initial information. I’ve now added more details about how I debugged the problem to that issue. Please let me know if anything is unclear or if you need further clarification.

[yanj-github]

[OF] When examined as a continuous sequence of timestamped audio samples of the audio stream, the 20ms test audio samples shall be a complete rendering of the source audio track and are rendered in increasing presentation time order.
Improved the algorithm so that audio data captured before the play() event not omitted from the calculation.
However, there are some starting and ending audio is missing and OF correctly reporting the missing audio part:
Starting and ending audio is missing is not OF issue, a new issue cta-wave/dpctf-tests#205 is raised for further diagnose and next action.

[OF] Audio start-up-delay: The start-up delay shall be sufficiently low.
Improved the algorithm so that audio data captured before the play() event not omitted from the calculation. Most of start up delay is pass now.

[OF] Audio: The playback duration shall match the duration of the CMAF Track.
Improved the algorithm so that audio data captured before the play() event not omitted from the calculation. Most of start up delay is pass now.

[yanj-github]

@jpiesing
Shall we make start_segment_num_tolerance in config.ini default value to 3 please?
How about then end_segment_num_tolerance?

[jpiesing]

@jpiesing Shall we make start_segment_num_tolerance in config.ini default value to 3 please?

Yes.

How about then end_segment_num_tolerance?

end_segment_num_tolerance is more sensitive with content providers and advertisers as adverts often have finish with words that are required by law or regulation.
We can look at what value would be needed to make a difference in the latest results but, in the earlier set, it was more than 3.

[yanj-github]

@wschidol do you have any comments on cta-wave/dpctf-tests#205 missing audio at the end please?