Sending 2 streams to FFmpeg from nodejs

Question

I am trying to send 2 ReadableStreams to FFmpeg from nodejs. I have tried using fluent-ffmpeg library to do this, but it only supports sending one stream for processing. Check here

My problem is: I have 2 incoming mono audio streams, I want to send them to ffmpeg to create a stereo stream, which I will then send to google's speech to text service, to generate a transcription.

I am successfully receiving both the mono streams to the nodejs server. How to utilize FFmpeg to merge them in realtime is still unclear, I could spawn a FFmpeg child process, but I'm not sure how to give 2 ReadableStreams as inputs and get the output as another stream? FFmpeg supports multiple input streams.

I can merge the 2 mono streams if they are in two separate files with this code.

const { spawn } = childProcess;
const ffmpeg = spawn('ffmpeg', [
  '-i', this.phoneAudioFile,
  '-i', this.micAudioFile,
  '-filter_complex', '[0:a][1:a]amerge=inputs=2[a]',
  '-map', '[a]',
  this.outputLosslessFile,
]);

How can I acheive the same using 2 streams instead of 2 files?

EDIT

• The incoming streams both have PCM audio data.
• This entire process runs on a linux Ubuntu server.
• The final output must be a wav file.

Answer 1

Assuming your source audio streams are regular PCM audio (such as what is most commonly found in WAV files), I would merge the streams internally in your application, and output a single stream to FFmpeg.

This can be done as simply as alternating which stream you read from, effectively interleaving the samples.

If your samples are 16-bit, then each sample is two bytes. So, your stream will look like this:

[LL][RR][LL][RR][LL][RR]

(where each LL is 2 bytes of a single sample for the left channel, and the same for RR )

If you're going to pipe this into FFmpeg, you'll need to set up the appropriate parameters for RAW PCM. Or, you can generate a WAV file header in your application as well.