Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture, receive, and RTP timestamp concept definitions & normative requirements for gUM/gDM #156

Merged
merged 19 commits into from
Oct 31, 2024
Merged

Capture, receive, and RTP timestamp concept definitions & normative requirements for gUM/gDM #156

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6f40605
Define captureTime.
handellm Sep 9, 2024
ae946a9
Goldplate
handellm Sep 11, 2024
2dc8783
Address comments from internal review.
handellm Sep 11, 2024
41da924
Updates after offline review
handellm Sep 20, 2024
069793f
Updates after TPAC joint Media/WebRTC session.
handellm Sep 27, 2024
88a2e13
Clarify nature and utility of timestamps.
handellm Sep 27, 2024
15e2399
Address reviewer comments.
handellm Sep 27, 2024
958a9bd
Update index.html
handellm Oct 7, 2024
c79520a
Update index.html
handellm Oct 7, 2024
4d9e139
Update index.html
handellm Oct 7, 2024
584ace5
Update index.html
handellm Oct 7, 2024
8d59ae2
Update index.html
handellm Oct 7, 2024
f5d3490
Update index.html
handellm Oct 7, 2024
d915bd5
Adress reviewer comments.
handellm Oct 7, 2024
a8fc811
Address reviewer comments.
handellm Oct 8, 2024
b5778b0
Clarify nature of presentation time, introduce offset in webcodecs vi…
handellm Oct 9, 2024
012628d
fix
handellm Oct 9, 2024
794cd2d
Update index.html
handellm Oct 10, 2024
fcafe1c
Add a note about presentation/capture timestamps being potentially th…
handellm Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 29 additions & 20 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1155,20 +1155,20 @@ <h2>Constrainable Properties</h2>
</table>
</section>
<section class="informative">
<h2>Timestamp concepts</h2>
<h2>Video timestamp concepts</h2>
<p>
Media flowing inside media stream tracks comprise of a sequence of media frames, where
Video media flowing inside media stream tracks comprise of a sequence of video frames, where
the frames are sampled from the media at instants spread out over time.
</p>
<p>
Each video frame has a <dfn class="export">presentation timestamp</dfn>
which is relative to the first frame appearing on the track.
</p>
<p>
Each frame may have a <dfn class="export">capture timestamp</dfn> representing
the instant it was captured. A source of frames can define how this timestamp is set, otherwise
it is unset.
</p>
<p>
Each video frame has a <dfn class="export">presentation timestamp</dfn>
which is relative to the first frame appearing on the track.
</p>
<p>
Each frame may have a <dfn class="export">receive timestamp</dfn> representing
the last received timestamp of packets used to produce this video frame was received in its entirety.
Expand All @@ -1193,30 +1193,39 @@ <h5>Members</h5>
<dl class="dictionary-members" data-link-for="VideoFrameMetadata" data-dfn-for="VideoFrameMetadata">
<dt><dfn><code>captureTime</code></dfn> of type <span class="idlMemberType">DOMHighResTimeStamp</span></dt>
<dd>
<p>
The capture timestamp of the frame relative to {{Performance}}.{{Performance/timeOrigin}}.
MUST be set to the [=capture timestamp=] of the source {{MediaStreamTrack}} video frame if set.
It is not [=map/exist|present=] if [=capture timestamp=] is unset.
<p>The capture timestamp of the frame relative to {{Performance}}.{{Performance/timeOrigin}}. It corresponds to
the [=capture timestamp=] of {{MediaStreamTrack}} video frames.
</p>
</dd>
<dt><dfn><code>receiveTime</code></dfn> of type <span class="idlMemberType">DOMHighResTimeStamp</span></dt>
<dd>
<p>
The receive time of the corresponding encoded frame relative to {{Performance}}.{{Performance/timeOrigin}}.
MUST be set to the [=receive timestamp=] of the source {{MediaStreamTrack}} video frame if set.
It is not [=map/exist|present=] if [=receive timestamp=] is unset.
</p>
<p>The receive time of the corresponding encoded frame relative to {{Performance}}.{{Performance/timeOrigin}}.
It corresponds to the [=receive timestamp=] of {{MediaStreamTrack}} video frames.</p>
</dd>
<dt><dfn><code>rtpTimestamp</code></dfn> of type <span class="idlMemberType">unsigned long</span></dt>
<dd>
<p>
The RTP timestamp of the corresponding encoded frame.
MUST be set to the [=RTP timestamp=] of the source {{MediaStreamTrack}} video frames if set.
It is not [=map/exist|present=] if [=receive timestamp=] is unset.
</p>
<p>The RTP timestamp of the corresponding encoded frame. It corresponds to [=RTP timestamp=] of
{{MediaStreamTrack}} video frames.</p>
</dd>
</dl>
</section>
<h3>Algorithms</h3>
The <dfn for=VideoFrame class="export">Initialize Video Frame Timestamps From Internal MediaStreamTrack Video Frame</dfn>
algorithm runs with |frame| as input.
<ol>
<li>Set {{VideoFrame/timestamp}} from [=presentation timestamp=].</li>
<li>Set {{VideoFrameMetadata/captureTime}} from [=capture timestamp=] if set.</li>
<li>Set {{VideoFrameMetadata/receiveTime}} from [=receive timestamp=] if set.</li>
<li>Set {{VideoFrameMetadata/rtpTimestamp}} from [=RTP timestamp=] if set.</li>
</ol>
The <dfn for=VideoFrame class="export">Copy Video Frame Timestamps To Internal MediaStreamTrack Video Frame</dfn>
algorithm runs with |frame| as input.
<ol>
<li>Set [=presentation timestamp=] from {{VideoFrame/timestamp}}.</li>
<li>Set [=capture timestamp=] from {{VideoFrameMetadata/captureTime}} if [=map/exist|present=].</li>
<li>Set [=receive timestamp=] from {{VideoFrameMetadata/receiveTime}} if [=map/exist|present=].</li>
<li>Set [=RTP timestamp=] from {{VideoFrameMetadata/rtpTimestamp}} if [=map/exist|present=].</li>
</ol>
</section>
<section>
<h3>Local video capture timestamps</h3>
Expand Down
Loading