Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jitter buffer improvements #55

Open
trevorirwin opened this issue Mar 17, 2020 · 1 comment
Open

Jitter buffer improvements #55

trevorirwin opened this issue Mar 17, 2020 · 1 comment

Comments

@trevorirwin
Copy link
Contributor

The very simple jitter buffer implemented in #54 is functional but very simple. There are many improvements that could still be made. Suggestions so far include:

  • Detecting network traffic delay and dynamically adjusting the buffer length
  • Replacing the time-based delay with sample-based delay
    • This would also allow for tests to be written against it without attempting to simulate time
  • Improving jitter buffer configurability. Right now, the desired delay is available as a property on each buffer object. A separate value is actually used privately. The private value is only updated when the jitter buffer "starts" delaying. A more elegant solution with more options should be on the table, especially if dynamic adjustment is made available.
@martindevans
Copy link
Owner

Detecting network traffic delay and dynamically adjusting the buffer length

The best approach to this would be to measure the standard deviation of elapsed time between packets arriving and set the buffer to some multiple of that.

Replacing the time-based delay with sample-based delay

This would be really nice for both testing and more exact timing (we're configuring everything with wall clock time, but really we don't care about that, just how many samples of time have elapsed in the audio system).

Dynamic Adjustment

This is surprisingly tricky to do. To expand or shrink the buffer you basically have two options:

  • Chain a resampler and resample the audio to the "wrong" rate but then play back at the normal rate. This drains the buffer faster/slower but also changes the pitch so you have to keep the rate change very low (5% or less is pretty much inaudible).
  • Modify the signal to stretch it (by repeating very small clips of it) or squash it (by cutting out very small clips).

@Meetsch Meetsch added this to the Voice Support milestone Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants