Codec negotiation and level IDs

[henbos]

If you are capable of level-id=X then you are also capable of level-id=Y where Y<X, assuming mimeType, profile-id, tier-flag and tx-mode is the same. I.e.

• If I can send H265 level-id=180 and receive level-id=156 that means I can sendrecv 156.
• If I can receive H265 level-id=156 and send level-id=180 that means I can sendrecv 156.

Today's filter seems to assume that every unique combination of codec, profile and level-id is included in the list:

If transceiver.direction is "sendonly" or "sendrecv", exclude any codecs not included in the list of implemented send codecs for kind.

If transceiver.direction is "recvonly" or "sendrecv", exclude any codecs not included in the list of implemented receive codecs for kind.

But in reality, user agents only list the highest level-id they support. Such as 186, rather than all possible values between 0 and 186 as separate codec values. This means you can offer one level-ID from your getCapabilities but this gets downgraded to a level-ID that is not in your getCapabilities with interesting consequences:

• What got negotiated (getParameters().codecs) is different from what was preferred (setCodecPreferences) or what was in getCapabilities.
• If sender capability is 186, but 180 was negotiated, can I still setParameters with 186 or do I have to change to 180?

[henbos]

@aboba You have pointed this (or something similar) out before when I didn't think about it. Does offering and answering 156 in the above example if direction=sendrecv make sense to you? (If direction is sendonly or recvonly we would still offer/answer the highest value we support in that direction, so this issue only becomes apparent when direction is sendrecv)

[henbos]

I see that the level-id way of thinking is consistent with our take on codec directionality as per https://github.com/ietf-wg-avtcore/draft-ietf-avtcore-hevc-webrtc/pull/29/files and we should just clarify the same applies here.

[aboba]

@henbos Yes. Note that for HEVC the "media format" doesn't include level-id.

[henbos]

What are the implications of "media format" not including level-id? That link says "Implementations SHOULD include this parameter within SDP" but does not mention it in the fmtp sentence?

Today in Chromium if I call RTCRtpSender.getCapabilities/RTCRtpReceiver.getCapabilities with

--enable-features=WebRtcAllowH265Send,WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled/

I do see the level-id in the codec.sdpFmtpLine values. In your view, is that a bug or a feature?

[henbos]

On the SDP negotiation side, it is my understanding that we know how to deal with level-ids and unit tests are being added in https://webrtc-review.googlesource.com/c/src/+/367322 (C++ layer, no WPTs yet, we can also do WPTs later).

However on the JS API side I think it might get confusing if setParameters rejects with values from getCapabilities?

[taste1981]

I would expect setParameters only select codecs as you get from getParameters(). What you get from getParameters() is related to the current direction of transceiver.

RtpSender.getCapabilities() now only returns the supported codecs when the transceiver is in the sendonly direction. It is not reporting capability for a sendrecv transceiver(unless we have a transceiver.getCapability() API).

[dontcallmedom-bot]

This issue had an associated resolution in WebRTC November 19 2024 meeting – 19 November 2024 (Issue #3020: Codec negotiation and level IDs):

RESOLUTION: Proposal A

[henbos]

@taste1981 yes you should only be able to pick from what was negotiated, we don't want to change that. The issue here is specifically about the level-id of a codec getting "downgraded" for your codec/payload type that was in fact accepted, see slides.