icecast audio.js play without buffering

Question

I have an icecast setup running on a server. The clients that will be connecting to it are tags in web pages, either through HTML5 or Flash. I am currently using audio.js to achieve this (specifically, the flash fallback).

The problem is, the audio is being played concurrently but separately with a stream of images. (It's a 10-fps jpeg stream.) I need the audio to match up as much as possible with the images. Unfortunately, the audio is sometimes as much as 7 seconds delayed before it starts playing.

Some information:

• The image stream cannot be delayed to match the audio. The audio must speed up to match the images.
• The icecast server config has <burst-on-connect> set to 0 to minimize latency.
• There is essentially no lag when playing via VLC (perhaps a few hundred ms, which is acceptable).

Put another way, when viewing the images and playing the audio via vlc, everything is sufficiently aligned. Unfortunately, using VLC is not an option in the endgame.

Since VLC has no lag, that tells me that the web browser (Chrome, firefox, IE) is buffering the audio before playing it.

The question: How do I prevent the web browser from buffering the audio? I want it to play immediately as soon as it has anything available. I'm currently using audio.js, but other similar technologies are acceptable.

Additional information: I've set audio.js to autoplay and preload=none.

Thanks for your help!

Answer 1

A buffer is always necessary. Networks are packet switched. Data comes in chunks, not continuously. In fact, there are many buffers:

• Capture buffer (at the sound card)
• Codec buffer (codecs work on a chunk of samples at a time)
• Network buffer to server
• Server-side buffer (typically very large, 10+ seconds)
• Network buffer to client
• Client buffer (typically 2-3 seconds)
• Client codec buffer
• Client sound device buffer

Each buffer adds latency, as you have noticed. The only buffer you really have control over is the server-side buffer, which is configured by the <burst-on-connect> setting. By setting the size of this buffer to a larger size, you can fill all downstream buffers very quickly, enabling an extremely fast start to playback. You have set this to zero, which means that the downstream buffers can only fill as fast as the data comes in from the encoder.

Client-side, you have absolutely no control over the buffering, and nor should you. Clients are free to implement a codec in whatever way they choose. Some codecs can begin streaming right away, and others can't. Some devices have to re-sample your audio to fit within their playback, and others don't.

What it sounds like you really want to do is synchronize a video stream and an audio stream. For that, you should be just streaming a video stream to begin with. Video is made to keep audio and video in sync. Icecast even supports streaming video in a few formats.