使用UDP和Python Server從Android到PC的音頻流中的噪聲

Question

我正在將UDP與Android App作為客戶端和Python服務器一起使用，將手機的麥克風輸入數據流式傳輸到PC。 它工作正常，沒有錯誤。

但是即使進行了很多調整，我在服務器端也會聽到很多噪音。 我想知道我的代碼有什么問題還是正常的？

客戶：

    public class MainActivity extends Activity {
    private Button startButton,stopButton;

    public byte[] buffer;
    public static DatagramSocket socket;
    private int port=8080;

    AudioRecord recorder;

    private int sampleRate = 44100 ; // 44100 for music
    private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;
    private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;
    int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
    private boolean status = true;


    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        startButton = (Button) findViewById (R.id.start_button);
        stopButton = (Button) findViewById (R.id.stop_button);

        startButton.setOnClickListener (startListener);
        stopButton.setOnClickListener (stopListener);

    }

    private final OnClickListener stopListener = new OnClickListener() {

        @Override
        public void onClick(View arg0) {
            status = false;
            recorder.release();
            Log.d("VS","Recorder released");
        }

    };

    private final OnClickListener startListener = new OnClickListener() {

        @Override
        public void onClick(View arg0) {
            status = true;
            startStreaming();
        }

    };

    public void startStreaming() {


        Thread streamThread = new Thread(new Runnable() {

            @Override
            public void run() {
                try {

                    DatagramSocket socket = new DatagramSocket();
                    Log.d("VS", "Socket Created");

                    byte[] buffer = new byte[minBufSize];

                    Log.d("VS","Buffer created of size " + minBufSize);
                    DatagramPacket packet;


                    Log.d("VS", "Address retrieved");

                    final InetAddress destination = InetAddress.getByName("10.0.0.2");
                    Log.d("VS", "Address retrieved");


                    recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,sampleRate,channelConfig,audioFormat,minBufSize*10);
                    Log.d("VS", "Recorder initialized");

                    recorder.startRecording();


                    while(status == true) {


                        //reading data from MIC into buffer
                        minBufSize = recorder.read(buffer, 0, buffer.length);

                        //putting buffer in the packet
                        packet = new DatagramPacket (buffer,buffer.length,destination,port);

                        socket.send(packet);

                        System.out.println("MinBufferSize: " +minBufSize);


                    }


                } catch(UnknownHostException e) {
                    Log.e("VS", "UnknownHostException",e);


                } catch (IOException e) {
                    e.printStackTrace();
                    Log.e("VS", ""+ e);
                }
            }

        });
        streamThread.start();
    }
}

服務器：

import pyaudio
import socket
from threading import Thread
import numpy as np
from matplotlib import pyplot as plt

frames = []

def udpStream(CHUNK):

    udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    udp.bind(("0.0.0.0", 8080))

    while True:
       # soundData, addr = udp.recvfrom(CHUNK)
        soundData, addr = udp.recvfrom(CHUNK * CHANNELS * 2)
        frames.append(soundData)
        print numpydata
        plt.plot(numpydata)
        plt.show()

    udp.close()

def play(stream, CHUNK):
    BUFFER = 10
    while True:
            if len(frames) == BUFFER:
                while True:
                    try:
                        stream.write(frames.pop(0), CHUNK)
                    except: 
                        pass

if __name__ == "__main__":
    FORMAT = pyaudio.paInt16
    CHUNK = 1024
    CHANNELS = 2
    RATE = 44100

    p = pyaudio.PyAudio()

    stream = p.open(format=FORMAT,
                    channels = CHANNELS,
                    rate = RATE,
                    output = True,
                    input=True,
                    frames_per_buffer = CHUNK,
                    )

    Ts = Thread(target = udpStream, args=(CHUNK,))
    Tp = Thread(target = play, args=(stream, CHUNK,))
    Ts.setDaemon(True)
    Tp.setDaemon(True)
    Ts.start()
    Tp.start()
    Ts.join()
    Tp.join()

Answer 1

您的代碼不包含用於處理UDP網絡傳輸速率抖動的音頻安全緩沖區。 WiFi上的UDP既不是媒體同步也不是可靠的傳輸，因此可能需要幾分之一秒的預填充安全緩沖區，以及一些用於平滑處理丟失的內容。

Answer 2

我能夠通過簡單得多的服務器設置使用您的Android應用代碼，並且效果很好。

import pyaudio
import socket

FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
CHUNK = 4096

udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.bind(("0.0.0.0", 5001))

audio = pyaudio.PyAudio()
stream = audio.open(format=FORMAT, channels=CHANNELS, rate=RATE, output=True, frames_per_buffer=CHUNK)

try:
    while True:
        data, addr = udp.recvfrom(CHUNK)
        stream.write(data)
except KeyboardInterrupt:
    pass

print('Shutting down')
udp.close()
stream.close()
audio.terminate()