使用Java混音器混合两个音频流

Question

I want to mix two streams of audio using a java mixer, and write the output into a file.

To do so, I capture the input of my two microphones by reading from the TargetDataLines of my microphone mixers:

    //microphone TargetDataLines
    TargetDataLine finalIn = null;
    TargetDataLine finalIn2 = null;

    Mixer.Info[] mixerInfos = AudioSystem.getMixerInfo();
    for (Mixer.Info info: mixerInfos){

        Mixer m = AudioSystem.getMixer(info);
        Line.Info[] lineInfos = m.getSourceLineInfo();
        System.out.println("****mixer name:" + info.getName());
        if(info.getName().equals("C920 [plughw:1,0]"))
            finalIn=(TargetDataLine) m.getLine(m.getTargetLineInfo()[0]);
        if(info.getName().equals("C920_1 [plughw:3,0]"))
            finalIn2=(TargetDataLine) m.getLine(m.getTargetLineInfo()[0]);

        for (Line.Info lineInfo:lineInfos){
            System.out.println ("\n source line info: "+lineInfo);
        }
        lineInfos = m.getTargetLineInfo();
        for (Line.Info lineInfo:lineInfos){
            System.out.println ("target line info: "+lineInfo);
        }

    }

    AudioFormat format = new AudioFormat(8000,16, 1, true,false);

    finalIn.open(format,2000);
    finalIn.start();
    finalIn2.open(format,2000);
    finalIn2.start();

    int numBytesRead;
    byte[] targetData = new byte[finalIn.getBufferSize() / 5];

    int numBytesRead2;
    byte[] targetData2 = new byte[finalIn2.getBufferSize() / 5];

    while (true) {
            numBytesRead = finalIn.read(targetData,0,targetData.length);
            numBytesRead2 = finalIn2.read(targetData2, 0, targetData2.length);

The resulting data captured from the microphones will be in targetData , targetData2 . Now I want to mix these two byte arrays containing my sound samples received from the microphone into a new byte array and I want to use another Java mixer for that .

To do so, I create two SourceDataLine and one TargetDataLine using the following code:

        DataLine.Info targetInfo = new DataLine.Info(TargetDataLine.class, format);
        DataLine.Info sourceInfo = new DataLine.Info(SourceDataLine.class, format);
        SourceDataLine sourceLine = (SourceDataLine) AudioSystem.getLine(sourceInfo);
        SourceDataLine sourceLine2 = (SourceDataLine) AudioSystem.getLine(sourceInfo);

        sourceLine.open(format,2000);
        sourceLine.start();

        sourceLine2.open(format,2000);
        sourceLine2.start();

        TargetDataLine targetLine = (TargetDataLine)        AudioSystem.getLine(targetInfo);
        targetLine.open(format,2000);
        targetLine.start();

and then inside the "while" I will add:

    byte[] result = new byte[finalIn.getBufferSize() / 5];

    while (true) {
            numBytesRead = finalIn.read(targetData,0,targetData.length);
            numBytesRead2 = finalIn2.read(targetData2, 0, targetData2.length);

            sourceLine.write(targetData, 0, numBytesRead);
            sourceLine.write(targetData2, 0, numBytesRead2);
            targetLine.read(result, 0, numBytesRead2);
    }

however, while the sound is actually played properly through my headphones, the result array stays empty. How can I solve this problem? What am I doing wrong?

Answer 1

I wonder if you have a misconception about what a Java Mixer does. I think this class was poorly named, because of my background experience as an audio engineer. In my mind, a mixer is where you combine various audio tracks. This is not what a Java Mixer does.

I think to do what you want to do, you need to read your two lines in parallel. For each, you will read a buffer of bytes, and will have to convert the bytes to PCM data. Then, you can add the PCM data together and convert that back to bytes and output it.

There is a good tutorial that shows how to get at the individual bytes/frames here . A short way down the page, there is a code example. Pay attention to the location of the comment:

// Here, do something useful with the audio data that's 
// now in the audioBytes array...

This example reads from an AudioInputStream, but the technique for getting at the individual bytes from TargetDataLines is not that different. Of course, you would be reading from both lines instead of from the single AIS. And the "useful thing" you would be doing is simply adding the PCM data on a per-frame basis, in parallel.

Once you have the summed the audio data, you could maybe use some code I wrote and posted at another site, that takes a PCM stream and writes it out as a wav file. Hopefully the discussion on that page is clear enough for you to follow.