Android's AEC and Noise Suppressor aren't working on some devices

Question

In working on an Android Unity app with VoIP functionality, we've discovered that on some devices (The Galaxy S6 specifically) the microphone will pick up the sounds output by the speaker, leading to a lot of static, distortion, and echo. Because of this I've started working on a native solution to take advantage of some of the AEC and NS features of Android.

I've written the following testbed, which works perfectly on some devices such as the Note 4. When the same app runs on the Galaxy S6, it's as if ARC and NS aren't being applied very well or at all.

public class MainActivity extends Activity
{
    private AudioRecord mAudioRecorder = null;
    private AudioTrack mStreamingPlayer = null;

    private Thread playbackThread = null;

    private boolean useVoip = true;

    int bufferSize;
    int chunkSize;

    private AutomaticGainControl agc;
    private NoiseSuppressor suppressor;
    private  AcousticEchoCanceler aec;

    private boolean manuallyAttachEffects = false;

    private void StartRecordingAudio()
    {
        int sampleRate = 44100;
        int channelConfig = AudioFormat.CHANNEL_IN_MONO;
        int audioFormat = AudioFormat.ENCODING_PCM_16BIT;

        chunkSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
        bufferSize = chunkSize * 10;

        int audioSource = MediaRecorder.AudioSource.MIC;

        if ( useVoip && !manuallyAttachEffects )
        {
            audioSource = MediaRecorder.AudioSource.VOICE_COMMUNICATION;
        }

        mAudioRecorder = new AudioRecord( audioSource, 
                                        sampleRate,
                                        channelConfig,
                                        audioFormat,
                                        bufferSize );

        if ( useVoip && manuallyAttachEffects )
        {
            SetupVoipEffects( mAudioRecorder.getAudioSessionId() );
        }

        mAudioRecorder.startRecording();

        mStreamingPlayer = new AudioTrack(  AudioManager.STREAM_MUSIC,
                                            sampleRate,
                                            AudioFormat.CHANNEL_OUT_MONO,
                                            audioFormat,
                                            bufferSize,
                                            AudioTrack.MODE_STREAM );

        mStreamingPlayer.play();

        playbackThread = new Thread(new Runnable()
        {
            @Override
            public void run() {
                readAudioData();
            }

        });

        playbackThread.start();
    }

    private void readAudioData()
    {
        Thread thisThread = Thread.currentThread();

        byte[] buffer = new byte[chunkSize];

        while ( !thisThread.isInterrupted() )
        {
            if ( mAudioRecorder == null )
                break;

            int bytesRead = mAudioRecorder.read(buffer, 0, chunkSize);
            if ( bytesRead > 0 )
            {
                if ( mStreamingPlayer == null )
                    break;

                mStreamingPlayer.write(buffer, 0, chunkSize);   
            }
        }
    }

    private void SetupVoipEffects( int sessionId )
    {
        if(AutomaticGainControl.isAvailable())
        {
            agc = AutomaticGainControl.create( sessionId );
            Log.d(LOG_TAG, "AGC is " + (agc.getEnabled()?"enabled":"disabled"));

            if ( !agc.getEnabled() )
            {
                agc.setEnabled(true);
                Log.d(LOG_TAG, "AGC is " + (agc.getEnabled()?"enabled":"disabled" +" after trying to enable"));
            }
        }

        if(NoiseSuppressor.isAvailable())
        {
            suppressor = NoiseSuppressor.create( sessionId );
            Log.d(LOG_TAG, "NS is " + (suppressor.getEnabled()?"enabled":"disabled"));

            if ( !suppressor.getEnabled() )
            {
                suppressor.setEnabled(true);
                Log.d(LOG_TAG, "NS is " + (suppressor.getEnabled()?"enabled":"disabled" +" after trying to disable"));  
            }

        }
        if(AcousticEchoCanceler.isAvailable())
        {
            aec = AcousticEchoCanceler.create( sessionId );
            Log.d(LOG_TAG, "AEC is " + (aec.getEnabled()?"enabled":"disabled"));

            if ( !aec.getEnabled() )
            {
                aec.setEnabled(true);
                Log.d(LOG_TAG, "AEC is " + (aec.getEnabled()?"enabled":"disabled" +" after trying to disable"));
            }
        }
    }
}

I've tried a lot of things such as using VOICE_COMMUNICATION for the AudioRecord, using MIC and manually creating the NoiseSuppressor and AcousticEchoCanceller, disabling/enabling NuPlayer/AwesomePlayer, and more. The devices show that AEC is supported and enabled, but it just doesn't seem to be happening. The problem completely goes away when using headphones, or using the handset instead of the speaker phone (so the mic can't pick up speaker sounds). Am I overlooking something huge here, or is it possible that there's a bug to be reported?

I wouldn't be suspicious if AEC and NS weren't working perfectly on other devices.

Answer 1

This is a known problem in Android. Hardware AEC is not reliable and it is not implemented/supported by lots of hardware even if they report it otherwise. I would recommend to deploy also a software AEC alongside with the hardware AEC to make this feature more stable in your software. The only problem is that software AEC is not so easy to accomplish correctly, so be prepared for a lots of tests.