WebAudio panner 无法与 WebRTC 音频 stream 一起正常工作

Question

I have an issue where my audio panner isn't properly panning with the given values.

Currently, if I set positionX.value to 1000, the audio plays as it was in the middle and not panned at all to the right channel.

Now if I set positionX.value to 0.5 or 0.9 or 1, the audio plays on the right channel, (even though not entirely, as I can still hear a bit on the left, more than usual).

I don't understand why it only works from -1 to 1, any number higher than that the audio goes back to the center. Any idea why? And I'm sure it should work with higher numbers because I have tested it before in a different situation and nothing says it shouldn't on the documentation.

peerConnection.onaddstream = (event) => {
    var panner = this.aContext.createPanner();
    panner.panningModel = 'HRTF';
    panner.distanceModel = 'inverse';
    panner.refDistance = 1;
    panner.maxDistance = 10000;
    panner.rolloffFactor = 1;
    panner.coneInnerAngle = 360;
    panner.coneOuterAngle = 0;
    panner.coneOuterGain = 0;
    
    // here is how Im setting the position, using -1 to 1 works, nothing greater tho
    panner.positionX.value = 10000;

    var source = this.aContext.createMediaStreamSource(event.stream);

    source.connect(panner);
    panner.connect(this.aContext.destination);

    const recvAudio = new Audio();
    recvAudio.srcObject = source.mediaStream;
    recvAudio.autoplay = true;
}

Answer 1

You need to set where ths listner is. Where it is determines the panning of your source node.

By default the listners position is (0,0,0) which is why you can here it when you set the source nodes with low values.

You can set where the listner is with:

var const listener = this.aContext.listener;
listener.setPosition(x,y,z);

Answer 2

Your recvAudio Audio element is diffusing the raw MediaStream directly, without the PannerNode affecting it.

    var source = this.aContext.createMediaStreamSource(event.stream);
///...
    recvAudio.srcObject = source.mediaStream;

In this snippet, source.mediaStream is exactly the same object as event.stream .

 document.querySelector("button").onclick = (evt) => { const context = new AudioContext(); // let's create an audio MediaStream from the AudioContext // in StackSnippets we can't use gUM... const event_stream = context.createMediaStreamDestination().stream; const source = context.createMediaStreamSource( event_stream ); console.log( "Same Object:", source.mediaStream === event_stream ); context.close(); }

 <button>test</button>

So when you get outside of the reference distance [-1 ~ 1], your AudioContext's output sound will get lower than the one of this Audio element, and you will have the impression that the PannerNode doesn't work anymore, because the Audio element's output covers it.

To fix it, the best is probably to not use an Audio element at all here, you don't really need it since you can let the AudioContext output the sound itself.

But if you really need to use an Audio element, instead of connecting the AudioPanner node to the context's destination, connect it to a MediaStreamDestinationNode and pass this latter as the srcObject of your Audio element:

//...
    panner.positionX.value = 10000;

    const source = this.aContext.createMediaStreamSource(event.stream);
    const destination = this.aContext.createMediaStreamDestination();
    source.connect(panner);
    panner.connect(destination);

    const recvAudio = new Audio();
    recvAudio.srcObject = destination.stream;
    recvAudio.autoplay = true;
//...

https://jsfiddle.net/do9xq681/