How to stream MP3 data via WebSockets with node.js and socket.io?

Question

I have problems streaming MP3 data via WebSocket with node.js and socket.io. Everything seems to work but decodeAudioData doesn't play fair with me.

This is my toy server:

var app = require('http').createServer(handler)
  , io = require('socket.io').listen(app)
  , fs = require('fs')

app.listen(8081);

function handler (req, res) {
    res.writeHead(200, {
        'Content-Type': 'text/html',
    });
    res.end('Hello, world!');
}

io.configure('development', function() {
  io.set('log level', 1);

  io.set('transports', [ 'websocket' ]);
});

io.sockets.on('connection', function (socket) {
    console.log('connection established');

    var readStream = fs.createReadStream("test.mp3", 
                                         {'flags': 'r',
                                          'encoding': 'binary', 
                                          'mode': 0666, 
                                          'bufferSize': 64 * 1024});
    readStream.on('data', function(data) {
        console.log(typeof data);
        console.log('sending chunk of data')
        socket.send(data);
    });

    socket.on('disconnect', function () {
        console.log('connection droped');
    });
});

console.log('Server running at http://127.0.0.1:8081/');

The client receives the data as type string but I want to feed the data to decodeAudioData and it seems it doesn't like strings. The call to decodeAudioData results in the following error message:

Uncaught Error: SYNTAX_ERR: DOM Exception 12

I think decodeAudioData needs the data stored in an ArrayBuffer. Is there a way to convert the data?

This is the client code:

<script src="http://127.0.0.1:8081/socket.io/socket.io.js"></script>
<script>
    var audioBuffer = null;
    var context = null;
    window.addEventListener('load', init, false);
    function init() {
        try {
            context = new webkitAudioContext();
        } catch(e) {
            alert('Web Audio API is not supported in this browser');
        }
    }

    function decodeHandler(buffer) {
        console.log(data);
    }

    var socket = io.connect('http://127.0.0.1:8081');
    socket.on('message', function (data) {
            // HERE IS THE PROBLEM
        context.decodeAudioData(data, decodeHandler, function(e) { console.log(e); });
    });
</script>

Answer 1

I've found a way to stream MP3 data via Websockets myself.

One problem was the chunk size of the MP3 data. It seems that the Web Audio API needs to be fed with valid MP3 chunks to be able to decode the data. Probably not surprising. In my demo app I provide a set of MP3 chunk files.

Also the quality of the audio is not perfect. I have some subtle glitches. I was able to improve that by sending larger chunks of MP3 data but there are still tiny crackles.

EDIT: I managed to improve the audio quality. It seems the Web Audio method decodeAudioData isn't really designed to decode continuos chunks of MP3 data.

Answer 2

In your case context.decodeAudioData expects an ArrayBuffer of the binary data, I would suggest converting your chunk to a base64 string, then to an ArrayBuffer client-side for the most predictable results. This script should be a good starting point for the client-side decode from base64 of the chunked data.

Adding a line with data = Base64Binary.decodeArrayBuffer(data); right after getting your data (base-64 encoded string) would do the trick...

Answer 3

It seems that socket.io still doesn't support Binary transfer. So websocket.io can be used here.

1. https://github.com/LearnBoost/socket.io/issues/511#issuecomment-2370129
2. https://github.com/LearnBoost/socket.io/issues/680#issuecomment-3083490