WebAudioAPI decodeAudioData() 在 iOS 14 Safari 上给出 null 错误

Question

I have an mp3 audio stream player that works well in every desktop browser, using MediaSourceExtensions with a fallback to WebAudioAPI for those browsers that do not support MSE. iOS Safari is one such browser, and should theoretically support mp3 decoding via the Web Audio API without issues.

I've been struggling to get iOS Safari to properly play the mp3 audio chunks that are being returned from the stream. So far, it's the only browser that seems to have issues and I can't for the life of me figure out what's going on. Sadly, there isn't a whole lot of information on corner cases like this and the other questions here on StackOverflow haven't been any help.

Here's the relevant part of my js code where things are getting hung up. It's a callback function for an async fetch() process that's grabbing the mp3 data from the stream.

async function pushStream(value) {
    // Web Audio streaming for browsers that don't support MSE
    if (usingWebAudio) {
        // convert the stream UInt8Array to an ArrayBuffer
        var dataBuffer = value.stream.buffer;
        // decode the raw mp3 chunks
        audCtx.decodeAudioData(dataBuffer, function(newData) {
            // add the decoded data to the buffer
            console.log("pushing new audio data to buffer");
            webAudioBuffer.push(newData);
            // if we have audio in the buffer, play it
            if (webAudioBuffer.length) {
                scheduleWebAudio();
            }
        }, function(e) {
            console.error(e);
        });

What I'm seeing is the error callback being fired and printing null: null as its error message (very helpful). Every so often, I will see the console print pushing new audio data to buffer , but this seems to only happen about once every few minutes while the stream is playing. Almost all the stream data is erroring out during the decode and the lack of useful error messages is preventing me from figuring out why.

As far as I can tell, iOS safari should support mp3 streams without any issues. It also should support the decodeAudioData() function. Most of the other answers I was able to find related to trying to play audio before the user interacts with the screen. In my case, I start the audio using a play button on the page so I don't believe that's the problem either.

One final thing, I'm developing on Windows and using the remotedebug iOS adapter . This could possibly be the reason why I'm not getting useful debug messages, however all other debug and error prints seem to work fine so I don't believe that's the case.

Thanks in advance for any help!

Answer 1

Unfortunately there is a bug in Safari which causes it to reject the decodeAudioData() promise with null. From my experience this happens in cases where it should actually reject the promise with an EncodingError instead.

The bug can be reproduced by asking Safari do decode an image. https://github.com/chrisguttandin/standardized-audio-context/blob/9c705bd2e5d8a774b93b07c3b203c8f343737988/test/expectation/safari/any/offline-audio-context-constructor.js#L648-L663

In general decodeAudioData() can only handle full files. It isn't capable of decoding a file in chunks. The WebCodecs API is meant to solve that but I guess it won't be available on iOS anytime soon.

However there is one trick that works with MP3s because of their internal structure. MP3s are built out of chunks themselves and any number of those chunks form a technically valid MP3. That means you can pre-process your data by making sure that each of the buffers that you pass on to decodeAudioData() begins and ends exactly at those internal chunk boundaries. The phonograph library does for example follow that principle.