Audio updates to the client


#1

We wanted to share with you some of the work that we have implemented into our audio.

Here’s a quick summary of the new features:

Max Frames Over Desired Jitter Buffer

  • Audio Mixer AND Interface now support a feature for “discarding” audio frames if the number of frames in the ring buffer grow to more than a specified amount over the desired jitter buffer. You can set this value in the interface preferences and in the mixer settings page. It defaults to 10 frames. This adds back an “undocumented design feature” (aka a bug that gave us a desirable effect) we had recently removed (aka fixed).

‘Desired Jitter Buffer Frames’ - in Audio Mixer Settings

  • We have had this option in the client for a while, we’ve now added it as a setting in the audio mixer. If you’ve enabled Dynamic Jitter Buffer calculation in the audio-mixer settings this value is ignored.

More Aggressive Dropping of Silent Packets

  • a couple of weeks ago we added a feature to drop silent packets in the audio-mixer if the frame count is more than our desired frame count. We’ve tweaked this logic to be more aggressive to allow us to get back to desired more quickly.

Reorganizing/Cleanup of Client Displayed Audio Stats

  • we made some tweaks to how audio stats are displayed which makes it easier to understand what detailed metrics are contributing to latency.

Goal of these Features? What do you get with these options? a.k.a. Why?

Default Behavior Closer to old behavior on bursty networks

  • If you deploy a standard stack without adjusting any options, you will get behavior that is almost identical to the old behavior from the last several months. This is mainly because the default audio-mixer settings have a desired jitter buffer set to 1 frame (same behavior as before) and if you ever burst up to over 10 frames, we will discard all packets till you get back to 1 frame. This mimics a bug that existed in the old audio rig buffer class that would discard all frames if we ever overflowed the ring buffer.

Two Knobs to Explicitly Control Jitter Buffers

  • In order to better experiment with tuning audio performance trade off between fidelity and latency, we can now dial in the desired minimum jitter buffer and the maximum jitter buffer allowed. This will allow us to play with finding the range which we find acceptable for this trade-off.

Automatic quick return to real time on bursty networks

  • on bursty networks, with noise cancellation turned on, the new more aggressive dropping of silent packets will more quickly return you to real time.

Other changes/Under the hood:

XxxxAudioStream classes

  • “Audio Streaming” behavior has been moved into a stream class and removed from the ring buffer class. Ring buffers are now just ring buffers. Streams have ring buffers instead of being derived from ring buffers and are used for inbound streams into the mixer and the client. These streams support common behavior for jitter buffer calculation and packet dropping.

@ZappoMan actually put this great write up together. I just used the powerful Copy and Paste function on the desktop :smile: