[bug] Too low quality score

[holzgeist]

Describe the bug
The quality score reported to the user through server events is too low in some cases.

To Reproduce

Steps to reproduce the behavior:

• Configure video capture to be 960x720 (4:3 720 preset)
• Connect to a room with certain Android devices that does not support natively capturing that resolution (e.g. Fairphone 5)
• Observe connection quality score go down to Good after a few seconds

This even happens on a local dev environment with low ping times/RTT.

Expected behavior
Excellent score due to local network being fast enough

Platform information

• Flutter version: 3.22.2
• Plugin version: 2.1.4
• Flutter target OS: Android
• Flutter target OS version: 13

Additional context
This is tightly related to webrtc-sdk/webrtc#133.
However, even with the partially fixing PR applied, the quality score is still too low.

The reason for this issue is that WebRTC Simulcast layer counts (https://github.com/webrtc-sdk/webrtc/blob/7ddfc43fcd2e4ba0548258a76fe88d49b34588c5/video/config/simulcast.cc#L82-L106) do not always match Livekit Flutter SDK layer counts https://github.com/livekit/client-sdk-flutter/blob/main/lib/src/utils.dart#L473-L477. Livekit server then assumes that bad network is the cause for the missing layers

Actual reasons for the mismatch:

• this SDK only considers max dimension, WebRTC considers pixel count (16:9 vs 4:3 comes into play too), and
• WebRTC potentially resizes video streams before calculating simulcast layers (see issue mentioned above)

[cloudwebrtc]

hey @holzgeist, Another possibility could be related to the DegradationPreference default setting. This PR has set the default value to balanced. Perhaps you can test your patch together with this PR to see if the video quality improves.

#556

[holzgeist]

Hi @cloudwebrtc
Thanks for the hint, I'll check it out.
I don't have much hope, because my added log output in scorer.go is pretty unambiguous:
"packetScore": 98.72154736490438, "bitrateScore": 100, "layerScore": 65

Full log line with context:

2024-07-15T16:26:36.994+0200	INFO	livekit.pub	connectionquality/scorer.go:413	Scorer:	{"room": "r1", "roomID": "RM_X6HwsbHncGLj", "participant": "p1", "pID": "PA_iXxtQHmwQwQZ", "remote": false, "trackID": "TR_VC34Ct5b9cB43t", "relayed": false, "mime": "video/vp8", "direction": "up", "expectedDistance": 1, "distanceWeight": 35, "packetScore": 98.72154736490438, "bitrateScore": 100, "layerScore": 65}

[holzgeist]

The PR you mentioned doesn't change anything unfortunately

[cloudwebrtc]

fixed https://github.com/livekit/client-sdk-flutter/releases/tag/v2.2.2