通过本地 TCP 套接字发送大字节数组太慢

Question

I'm streaming my desktop from one device to another by using DirectX desktop duplication on the sending device, copying the datastream of the resulting bitmap to a byte array, and then sending that byte array over a socket to a locally connected device over a WiFi connection.

The other device receives the byte array, copies it to a datastream to create a bitmap from it and then renders that to a window.

The duplicating/rendering only takes <20ms.

The time it takes to send each byte array is around 900ms - 1400ms. But if I connect to the socket as server/client on the same device, it's very faster. So the issue is with actually sending it to the other connected device.

Ideally I'd want this send/receive process to take <100ms.

If I send simple things like a single int, it gets received instantly.

I should also note the DataStream object is an array of colour data per pixel, so this is a constant size per resolution (ARGB 1920x1080 = 8294400 bytes).

The wifi adapter of the device I'm sending from has max link speed of 400/400 Mbps and the receiving device is 240/240, so I don't think this is the problem either.

Here is my code for socket:

using System;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Net.Sockets;

namespace StreamSocket {
    public class Instance {
        private Socket Host;
        private Socket Socket;
        
        public string InstanceType { get; set; }
        public int BufferSize { get; set; }
        public bool IsConnected { get { return Socket != null && Socket.Connected; } }

        public event EventHandler<EventArgs> Connected = delegate { };
        public event EventHandler<EventArgs> Disconnected = delegate { };
        public event EventHandler<EventArgs> ConnectionFailed = delegate { };
        public event EventHandler<DataReceivedEventArgs> DataReceived = delegate { };

        public class DataReceivedEventArgs : EventArgs {
            public int Code { get; private set; }
            public byte[] Data { get; private set; }

            public DataReceivedEventArgs(int code, byte[] data) {
                Code = code; Data = data;
            }
        }

        public void Start(int bufferSize, string ip = "") {
            BufferSize = bufferSize;

            if(ip == string.Empty) { //Start as Server
                Host = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                Host.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                Host.Bind(new IPEndPoint(IPAddress.Any, 8080));
                Host.Listen(1);
                Host.BeginAccept(new AsyncCallback(AcceptCallback), Host);
            } else { //Start as Client
                Socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                Socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                Socket.BeginConnect(ip, 8080, new AsyncCallback(ConnectCallback), Socket);
            }
        }

        public void Disconnect() {
            if(Socket == null) { return; }

            Disconnected(this, new EventArgs());
            if(Host != null) {
                if(Socket != null) {
                    Socket.Disconnect(true);
                }
                Host.BeginAccept(new AsyncCallback(AcceptCallback), Host);
            } else {
                Socket.Close();
            }
        }

        private void AcceptCallback(IAsyncResult result) {
            Socket = ((Socket)result.AsyncState).EndAccept(result);
            Connected(this, new EventArgs());
            BeginReceive();
        }

        private void ConnectCallback(IAsyncResult result) {
            try {
                Socket.EndConnect(result);
                Connected(this, new EventArgs());
                BeginReceive();
            } catch {
                ConnectionFailed(this, new EventArgs());
            }
        }

        public void Send(byte[] data) {
            if(!IsConnected) { return; }

            try {
                Socket.BeginSend(data, 0, data.Length, 0, new AsyncCallback(SendCallback), Socket);
            } catch { Disconnect(); }
        }

        private void SendCallback(IAsyncResult result) {
            try {
                Socket.EndSend(result);
            } catch { Disconnect(); }
        }

        private void BeginReceive() {
            if(!IsConnected) { return; }

            SocketData data = new SocketData() { Buffer = new byte[BufferSize] };
            Socket.BeginReceive(data.Buffer, 0, BufferSize, 0, new AsyncCallback(ReceiveCallback), data);
        }

        private void ReceiveCallback(IAsyncResult result) {
            if(!IsConnected) { return; }

            try {
                SocketData data = (SocketData)result.AsyncState;
                int bytesRead = Socket.EndReceive(result);
                if(bytesRead > 0) {
                    if(data.Length == 0) {
                        //Get headers from start of first buffer
                        using(MemoryStream ms = new MemoryStream(data.Buffer)) {
                            using(BinaryReader br = new BinaryReader(ms)) {
                                data.Code = br.ReadByte();
                                data.Length = br.ReadInt32();
                            }
                        }
                        //Write the rest of the buffer
                        if(data.Buffer.Length > sizeof(byte) + sizeof(int)) {
                            data.Stream.Write(data.Buffer, sizeof(byte) + sizeof(int), bytesRead - (sizeof(byte) + sizeof(int)));
                        }
                    } else {
                        //Writer buffer chunk that doesn't start with header
                        data.Stream.Write(data.Buffer, 0, bytesRead);
                    }

                    if(data.Stream.Length != data.Length) {
                        //Expected to receive more chunks
                        Socket.BeginReceive(data.Buffer, 0, BufferSize, 0, new AsyncCallback(ReceiveCallback), data);
                    } else {
                        //All data received
                        DataReceived(this, new DataReceivedEventArgs(data.Code, data.Stream.ToArray()));
                        data.Stream.Dispose();
                        
                        BeginReceive();
                    }
                } else {
                    Disconnect();
                }
            } catch { Disconnect(); }
        }
    }
}

The above is used by 2 other classes Server/Client that start instance of socket and receive the events like when receiving data:

private void Client_DataReceived(object sender, Instance.DataReceivedEventArgs e) {
    switch((Code)e.Code) {
        case Code.BitmapReceived:
            //This sends the result to the DX Renderer
            BytesToBitmapData(e.Data);
            break;
    }
}

Here are the functions for converting between BitmapData & byte array: (These only take around <10ms so I don't think they're a problem)

public static byte[] BitmapDataToBytes(SerialCode code, BitmapData bitmapData) {
    using(MemoryStream ms1 = new MemoryStream()) {
        using(BinaryWriter bw1 = new BinaryWriter(ms1)) {
            bw1.Write((byte)code); //Header - Code
            bw1.Write((sizeof(int) * 4) + bitmapData.Data.Length); //Header - Length
            //Body:
            bw1.Write(bitmapData.Size.Width);
            bw1.Write(bitmapData.Size.Height);
            bw1.Write(bitmapData.Pitch);
            bw1.Write(bitmapData.Data.Length);
            bw1.Write(bitmapData.Data);

            return ms1.ToArray();
        }
    }
}

public static BitmapData BytesToBitmapData(byte[] bytes) {
    using(MemoryStream ms = new MemoryStream(bytes)) {
        using(BinaryReader br = new BinaryReader(ms)) {
            Size size = new Size(br.ReadInt32(), br.ReadInt32());
            int pitch = br.ReadInt32();
            int length = br.ReadInt32();
            byte[] data = br.ReadBytes(length);

            return new BitmapData() { Size = size, Pitch = pitch, Data = data };
        }
    }
}

I assume the issue may be with efficiency of my socket code or a limitation of TCP? What could I do to improve the speed of sending large byte arrays?

Answer 1

240 Mbit/s indicated speed could move around 240/2 *95% = 14 MB/s (/2 for Wi-Fi overhead, 95% for TCP over IPv4 overhead), roughly 1.7 screens worth or 580 ms/screen - provided that there's no network congestion by nearby networks (or Bluetooth or...).

You need to either increase the link speed [*1] - Gigabit Ethernet is appr. 8 times faster - or decrease the amount of data by compressing (there are faster compressors around) or smart caching (only transmit changed pixels). For arbitrary video, a video codec like H.264 comes to mind which most GPUs or a beefy CPU could compress in realtime.

Btw, HDMI doesn't use a transmission speed of up to 48 Gbit/s without a cause.

[*1] The lower speed counts as you can't transmit data faster than it an be received.