Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BWE increases slowly when audio packets are sent before video packets. #5

Merged
merged 2 commits into from
Apr 28, 2024

Conversation

giangndm
Copy link

This PR have some changes:

For more clearly, I will explain what issues lead to this PR.

Problem

When an audio stream starts before a video stream, the Bitrate Estimator (BWE) calculates the initial bitrate based on the audio packets. This results in a very low estimated bitrate, typically around 40kbps. When the video stream starts, the BWE struggles to adapt to the increased bandwidth requirements, leading to a slow increase in bitrate. This can cause poor video quality.

        let sample_estimate_bps = sample_estimate.as_f64();
        let estimate_bps = estimate.as_f64();
        // Define the sample uncertainty as a function of how far away it is from the
        // current estimate. With low values of uncertainty_symmetry_cap_ we add more
        // uncertainty to increases than to decreases. For higher values we approach
        // symmetry.
        let sample_uncertainty =
            scale * (estimate_bps - sample_estimate_bps).abs() / (estimate_bps.max(25_000.0));
        let sample_var = sample_uncertainty.powf(2.0);

        // Update a bayesian estimate of the rate, weighting it lower if the sample
        // uncertainty is large.
        // The bitrate estimate uncertainty is increased with each update to model
        // that the bitrate changes over time.
        let pred_bitrate_estimate_var = self.estimate_var + 5.0;
        let mut new_estimate = (sample_var * estimate_bps
            + pred_bitrate_estimate_var * sample_estimate_bps)
            / (sample_var + pred_bitrate_estimate_var);
        new_estimate = new_estimate.max(ESTIMATE_FLOOR.as_f64());
        self.estimate = Some(Bitrate::bps(new_estimate.ceil() as u64));
        self.estimate_var =
            (sample_var * pred_bitrate_estimate_var) / (sample_var + pred_bitrate_estimate_var);

https://github.com/algesten/str0m/blob/a3e0f13744ef8b0681bd34b97298dbe72085878a/src/packet/bwe/acked_bitrate_estimator.rs#L76-91

In my local testing, I observed that the sample_var value was around 2000 and the self.estimate_var value was around 20. As a result, the BWE increased very slowly. For instance, if the estimated bitrate was 400 and the previous bitrate was 40, the BWE would only increase to 43.6, which is a mere 1% increase. After my configuration of a 2-second warm-up time ends, the video bitrate drops dramatically to around 80 kbps, causing video freezing and very poor quality.

Solution

This PR introduces a BWE reset API, which allows the BWE to be reset when the server starts sending video packets after an initial period of audio-only packets. This enables the BWE to quickly adapt to the changed bandwidth requirements, ensuring a smoother and more accurate bitrate estimation.

@giangndm giangndm force-pushed the fix-bwe-slow-increase-with-audio-first branch from 8097815 to 4a0a78e Compare April 26, 2024 08:23
@giangndm giangndm merged commit 0996cfb into 8xFF:main Apr 28, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant