How do you decode a audio file encoded as a byte string into a numpy array?

Question

Currently, I have project that is a React.js frontend paired up with a Flask backend. I am using the frontend to collect short audio tracks from the microphone and feeding that data to my backend for processing. Right now, the frontend sends the microphone data as a base64string in a POST request, and my backend needs to decode and save the data to the disk at "temp.wav" before loading the data back in as a python ndarray with librosa. For privacy and efficiency reasons, I don't want to have to save the file locally before loading it back into memory, but I can't seem to figure out how to load the translate the data from the byte64 representation to the numpy.ndarray that librosa outputs.

Here is a snipet of the frontend code:

let blob = new Blob(audio.audioChunks, {type: 'audio/wav'});
const reader = new FileReader();
reader.readAsDataURL(blob);
reader.onload = () => {
  let base64data = reader.result.split(',')[1];
  fetch('/api/task', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ message: base64data })
  }).then(res => res.json()).then(data =>{
    console.log(data)
  });
};

and my Flask backend code:

@app.route('/api/task', methods=['POST'])
def hello_world(): #just a filler name
    content = request.get_json()
    ans = base64.b64decode(bytes(content["message"], 'utf-8'))

    with open("temp.wav", "wb") as fh:
        fh.write(ans)

    audio_input, _ = librosa.load("temp.wav", sr=16000)
    #do something with audio_input

I really need the backend to be Flask because I need access to different Python tools that aren't available in other languages.

Answer 1

You'll use librosa.stream instead of librosa.load .

https://librosa.org/doc/main/generated/librosa.stream.html