播放实时音频流 - html5

Question

I have a desktop application which streams raw PCM data to my browser over a websocket connection. The stream looks like this ...\\\\x00\\\\x00\\\\x02\\\\x00\\\\x01\\\\x00\\\\x00\\\\x00\\\\x01\\\\x00\\\\xff\\\\xff\\\\xff\\\\xff\\\\... .

The question is simple: can I play such a stream in HTML with the Web Audio API / WebRTC / ...?

Any suggestions are very welcome!

code edit

This code plays noise, randomly generated:

function myPCMSource() { 
    return Math.random() * 2 - 3;
}

var audioContext;

try {
    window.AudioContext = window.AudioContext || window.webkitAudioContext;
    audioContext = new AudioContext();
} catch(e) {
    alert('Web Audio API is not supported in this browser');
}

var bufferSize = 4096;
var myPCMProcessingNode = audioContext.createScriptProcessor(bufferSize, 1, 1);
myPCMProcessingNode.onaudioprocess = function(e) {
    var output = e.outputBuffer.getChannelData(0);
    for (var i = 0; i < bufferSize; i++) {
     output[i] = myPCMSource(); 
 }
}

So changing the myPCMSource() to the websocket stream input, should make it work somehow. But it doesn't. I don't get any errors, but the API is not playing any sound nor noise.

Answer 1

Use a ScriptProcessorNode, but be aware that if there is too much load on the main thread (the thread that runs your javascript, draws the screen, etc.), it will glitch.

Also, your PCM stream is probably in int16, and the Web Audio API works in terms of float32. Convert it like so:

output_float[i] = (input_int16[i] / 32767);

that is, go from a [0; 65535] range to a [-1.0; 1.0] range.

EDIT I was using output_float[i] = (input_int16[i] / 32767 - 1); , this article shows that you should use output_float[i] = (input_int16[i] / 32767); . Now it's working fine!

Answer 2

Just for the record, the ScriptProcessorNode is deprecated . See the MDN article for details. The feature was replaced by AudioWorklets and the AudioWorkletNode interface .

In short, a ScriptProcessorNode runs outside of the browser's internal audio thread, which creates at least on frame (128 samples) of latency. Worse, the ScriptProcessorNode often fails to respond quickly enough, if the thread is busy, so will just randomly drop the ball every so often.

Worklets are basically task-specific workers that run in one of the browsers internal threads (paint, layout, audio etc). Audio worklets run in the audio thread, and implement the guts of custom audio nodes, which are then exposed through the WebAudio API as normal.

Note: You are also able to run WebAssembly inside worklets to handle the processing.

The solution provided above is still useful, as the basic idea holds, but it would ideally use an audio worklet.