Sending large byte array over local TCP socket is too slow

Question

I'm streaming my desktop from one device to another by using DirectX desktop duplication on the sending device, copying the datastream of the resulting bitmap to a byte array, and then sending that byte array over a socket to a locally connected device over a WiFi connection.

The other device receives the byte array, copies it to a datastream to create a bitmap from it and then renders that to a window.

The duplicating/rendering only takes <20ms.

The time it takes to send each byte array is around 900ms - 1400ms. But if I connect to the socket as server/client on the same device, it's very faster. So the issue is with actually sending it to the other connected device.

Ideally I'd want this send/receive process to take <100ms.

If I send simple things like a single int, it gets received instantly.

I should also note the DataStream object is an array of colour data per pixel, so this is a constant size per resolution (ARGB 1920x1080 = 8294400 bytes).

The wifi adapter of the device I'm sending from has max link speed of 400/400 Mbps and the receiving device is 240/240, so I don't think this is the problem either.

Here is my code for socket:

using System;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Net.Sockets;

namespace StreamSocket {
    public class Instance {
        private Socket Host;
        private Socket Socket;
        
        public string InstanceType { get; set; }
        public int BufferSize { get; set; }
        public bool IsConnected { get { return Socket != null && Socket.Connected; } }

        public event EventHandler<EventArgs> Connected = delegate { };
        public event EventHandler<EventArgs> Disconnected = delegate { };
        public event EventHandler<EventArgs> ConnectionFailed = delegate { };
        public event EventHandler<DataReceivedEventArgs> DataReceived = delegate { };

        public class DataReceivedEventArgs : EventArgs {
            public int Code { get; private set; }
            public byte[] Data { get; private set; }

            public DataReceivedEventArgs(int code, byte[] data) {
                Code = code; Data = data;
            }
        }

        public void Start(int bufferSize, string ip = "") {
            BufferSize = bufferSize;

            if(ip == string.Empty) { //Start as Server
                Host = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                Host.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                Host.Bind(new IPEndPoint(IPAddress.Any, 8080));
                Host.Listen(1);
                Host.BeginAccept(new AsyncCallback(AcceptCallback), Host);
            } else { //Start as Client
                Socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
                Socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, 1);
                Socket.BeginConnect(ip, 8080, new AsyncCallback(ConnectCallback), Socket);
            }
        }

        public void Disconnect() {
            if(Socket == null) { return; }

            Disconnected(this, new EventArgs());
            if(Host != null) {
                if(Socket != null) {
                    Socket.Disconnect(true);
                }
                Host.BeginAccept(new AsyncCallback(AcceptCallback), Host);
            } else {
                Socket.Close();
            }
        }

        private void AcceptCallback(IAsyncResult result) {
            Socket = ((Socket)result.AsyncState).EndAccept(result);
            Connected(this, new EventArgs());
            BeginReceive();
        }

        private void ConnectCallback(IAsyncResult result) {
            try {
                Socket.EndConnect(result);
                Connected(this, new EventArgs());
                BeginReceive();
            } catch {
                ConnectionFailed(this, new EventArgs());
            }
        }

        public void Send(byte[] data) {
            if(!IsConnected) { return; }

            try {
                Socket.BeginSend(data, 0, data.Length, 0, new AsyncCallback(SendCallback), Socket);
            } catch { Disconnect(); }
        }

        private void SendCallback(IAsyncResult result) {
            try {
                Socket.EndSend(result);
            } catch { Disconnect(); }
        }

        private void BeginReceive() {
            if(!IsConnected) { return; }

            SocketData data = new SocketData() { Buffer = new byte[BufferSize] };
            Socket.BeginReceive(data.Buffer, 0, BufferSize, 0, new AsyncCallback(ReceiveCallback), data);
        }

        private void ReceiveCallback(IAsyncResult result) {
            if(!IsConnected) { return; }

            try {
                SocketData data = (SocketData)result.AsyncState;
                int bytesRead = Socket.EndReceive(result);
                if(bytesRead > 0) {
                    if(data.Length == 0) {
                        //Get headers from start of first buffer
                        using(MemoryStream ms = new MemoryStream(data.Buffer)) {
                            using(BinaryReader br = new BinaryReader(ms)) {
                                data.Code = br.ReadByte();
                                data.Length = br.ReadInt32();
                            }
                        }
                        //Write the rest of the buffer
                        if(data.Buffer.Length > sizeof(byte) + sizeof(int)) {
                            data.Stream.Write(data.Buffer, sizeof(byte) + sizeof(int), bytesRead - (sizeof(byte) + sizeof(int)));
                        }
                    } else {
                        //Writer buffer chunk that doesn't start with header
                        data.Stream.Write(data.Buffer, 0, bytesRead);
                    }

                    if(data.Stream.Length != data.Length) {
                        //Expected to receive more chunks
                        Socket.BeginReceive(data.Buffer, 0, BufferSize, 0, new AsyncCallback(ReceiveCallback), data);
                    } else {
                        //All data received
                        DataReceived(this, new DataReceivedEventArgs(data.Code, data.Stream.ToArray()));
                        data.Stream.Dispose();
                        
                        BeginReceive();
                    }
                } else {
                    Disconnect();
                }
            } catch { Disconnect(); }
        }
    }
}

The above is used by 2 other classes Server/Client that start instance of socket and receive the events like when receiving data:

private void Client_DataReceived(object sender, Instance.DataReceivedEventArgs e) {
    switch((Code)e.Code) {
        case Code.BitmapReceived:
            //This sends the result to the DX Renderer
            BytesToBitmapData(e.Data);
            break;
    }
}

Here are the functions for converting between BitmapData & byte array: (These only take around <10ms so I don't think they're a problem)

public static byte[] BitmapDataToBytes(SerialCode code, BitmapData bitmapData) {
    using(MemoryStream ms1 = new MemoryStream()) {
        using(BinaryWriter bw1 = new BinaryWriter(ms1)) {
            bw1.Write((byte)code); //Header - Code
            bw1.Write((sizeof(int) * 4) + bitmapData.Data.Length); //Header - Length
            //Body:
            bw1.Write(bitmapData.Size.Width);
            bw1.Write(bitmapData.Size.Height);
            bw1.Write(bitmapData.Pitch);
            bw1.Write(bitmapData.Data.Length);
            bw1.Write(bitmapData.Data);

            return ms1.ToArray();
        }
    }
}

public static BitmapData BytesToBitmapData(byte[] bytes) {
    using(MemoryStream ms = new MemoryStream(bytes)) {
        using(BinaryReader br = new BinaryReader(ms)) {
            Size size = new Size(br.ReadInt32(), br.ReadInt32());
            int pitch = br.ReadInt32();
            int length = br.ReadInt32();
            byte[] data = br.ReadBytes(length);

            return new BitmapData() { Size = size, Pitch = pitch, Data = data };
        }
    }
}

I assume the issue may be with efficiency of my socket code or a limitation of TCP? What could I do to improve the speed of sending large byte arrays?

Answer 1

240 Mbit/s indicated speed could move around 240/2 *95% = 14 MB/s (/2 for Wi-Fi overhead, 95% for TCP over IPv4 overhead), roughly 1.7 screens worth or 580 ms/screen - provided that there's no network congestion by nearby networks (or Bluetooth or...).

You need to either increase the link speed [*1] - Gigabit Ethernet is appr. 8 times faster - or decrease the amount of data by compressing (there are faster compressors around) or smart caching (only transmit changed pixels). For arbitrary video, a video codec like H.264 comes to mind which most GPUs or a beefy CPU could compress in realtime.

Btw, HDMI doesn't use a transmission speed of up to 48 Gbit/s without a cause.

[*1] The lower speed counts as you can't transmit data faster than it an be received.