UDP视频流C＃

Question

我使用UDP进行视频流传输，但是由于帧的大小大于udp max size，因此我必须将其划分为若干段主数组并发送一些数据包。 没关系，但是它是udp，因此接收到的数组通常不按在客户端的顺序排列（类似这样：第一帧的第一段，第二帧的第三段等。）。 我可以对段进行编号，但是经常会收到错误帧中的段（第一帧的第一段，第二帧的第二段，第二帧的第三段）。 发送：

while(true){
int c = CvInvoke.cvWaitKey(33);
if (c == 27)
break;
    Byte[] byteData1 = rgb32Image.Bytes;
    Byte[] byte1 = new Byte[65000];
    Byte[] byte2 = new Byte[65000];
    Byte[] byte3 = new Byte[65000];
    Byte[] byte4 = new Byte[65000];
    byte1[0] = 1;
    byte2[0] = 2;
    byte3[0] = 3;
    byte4[0] = 4;
    System.Buffer.BlockCopy(byteData1, 0, byte1, 1, 64999);
    obj.socket.Send(byte1, byte1.Length);
    System.Buffer.BlockCopy(byteData1, 64999, byte2, 1, 64999);
    obj.socket.Send(byte2, byte2.Length);
    System.Buffer.BlockCopy(byteData1, 64999 * 2, byte3, 1, 64999);
    obj.socket.Send(byte3, byte3.Length);
    System.Buffer.BlockCopy(byteData1, 64999 * 3, byte4, 1, byteData1.Length - (3 * 64999);
    obj.socket.Send(byte4, byte4.Length);
}

接收：

VideoReceive(IAsyncResult ar)
{
StateObject stateobj = (StateObject)ar.AsyncState;
UdpClient client = stateobj.workSocket;
IPEndPoint senderIPEndPoint = new IPEndPoint(IPAddress.Any, 0);
Byte[] byte1 = client.EndReceive(ar, ref senderIPEndPoint);
Byte[] byte2 = client.Receive(ref senderIPEndPoint);
Byte[] byte3 = client.Receive(ref senderIPEndPoint);
Byte[] byte4 = client.Receive(ref senderIPEndPoint);
switch (byte1[0])
{
case 1:
System.Buffer.BlockCopy(byte1, 1, buffer, 0, 64999);
break;
case 2:
System.Buffer.BlockCopy(byte1, 1, buffer, 64999, 64999);
break;
case 3:
System.Buffer.BlockCopy(byte1, 1, buffer, 64999 * 2, 64999);
break;
case 4:
System.Buffer.BlockCopy(byte1, 1, buffer, 64999 * 3, 64999);
break;
}
switch (byte2[0])
{
case 1:
System.Buffer.BlockCopy(byte2, 1, buffer, 0, 64999);
break;
case 2:
System.Buffer.BlockCopy(byte2, 1, buffer, 64999, 64999);
break;
case 3:
System.Buffer.BlockCopy(byte2, 1, buffer, 64999 * 2, 64999);
break;
case 4:
System.Buffer.BlockCopy(byte2, 1, buffer, 64999 * 3, 64999);
break;
}
switch (byte3[0])
{
case 1:
System.Buffer.BlockCopy(byte3, 1, buffer, 0, 64999);
break;
case 2:
System.Buffer.BlockCopy(byte3, 1, buffer, 64999, 64999);
break;
case 3:
System.Buffer.BlockCopy(byte3, 1, buffer, 64999 * 2, 64999);
break;
case 4:
System.Buffer.BlockCopy(byte3, 1, buffer, 64999 * 3, 64999);
break;
}
switch (byte4[0])
{
case 1:
System.Buffer.BlockCopy(byte4, 1, buffer, 0, 64999);
break;
case 2:
System.Buffer.BlockCopy(byte4, 1, buffer, 64999, 64999);
break;
case 3:
System.Buffer.BlockCopy(byte4, 1, buffer, 64999 * 2, 64999);
break;
case 4:
System.Buffer.BlockCopy(byte4, 1, buffer, 64999 * 3, 64999);
break;
}

stateobj.workSocket.BeginReceive(new System.AsyncCallback(VideoReceive), stateobj);
mask2.Bytes = buffer;
pictureBox1.BeginInvoke(new MethodInvoker(() => { pictureBox1.Image = mask2.ToBitmap(); }));

}

那么，如何通过udp套接字流视频（没有任何其他库）？ PS：您可以通过代码或算法来回答。

Answer 1

我前一阵子做了这件事，我的解决方案是将图像分成几列，然后每列发送一次。 最初，我担心会有过多的帧撕裂，但是如果使用高fps的摄像头和流，它会很好地工作。

发件人：

if(imageAvailable) {
    try {
       image = //get image (I am using a BufferedImage image)
               // here I sent the image size through a tcp connection
       ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
       DataOutputStream stream = new DataOutputStream(byteStream);
       for(int a = 0; a < image.getWidth(); a++) {
          stream.writeInt(a);
          for(int b = 0; b < image.getHeight(); b++) {
               int rgb = image.getRGB(a, b);
               byte red = (byte) ((rgb >> 16) & 0x000000FF);
               byte green = (byte) ((rgb >> 8 ) & 0x000000FF);
               byte blue = (byte) ((rgb) & 0x000000FF);
               stream.writeByte(red);
               stream.writeByte(green);
               stream.writeByte(blue);
           }
                DatagramPacket packet = new DatagramPacket(byteStream.toByteArray(), 0,   
                                       byteStream.toByteArray().length, dsAddresses); //dsAddresses is the ip of the client.
                udpSocket.send(packet);
       }
       byteStream = new ByteArrayOutputStream();
       stream = new DataOutputStream(byteStream);
   }

接收器：

 public class DataReceiver implements Runnable {
        private DatagramSocket socket;
        byte [][][] packets;
        public DataReceiver(DatagramSocket socket) {
             this.socket = socket;
             packets = new byte[640][480][3];
        }

        @Override
        public void run() {
        while(true) {
              DatagramPacket packet = new DatagramPacket(new byte[1444], 1444);
              try {
                    socket.receive(packet);
              } catch (IOException e) {
                  e.printStackTrace();
              }
              try{
                 byte[] data = packet.getData();
                 DataInputStream inputStream = new DataInputStream(new          
                                               ByteArrayInputStream(data));
                 int colum = inputStream.readInt();
                 for (int a = 0; a < packets[0].length; a++){
                     packets[colum][a][0] = inputStream.readByte();
                     packets[colum][a][1] = inputStream.readByte() ;
                     packets[colum][a][2] = inputStream.readByte();
                 }
                 }catch(Exception e){
                    e.printStackTrace();
                 }

         }
         }
    public byte[][][] getData(){
         return packets;
    }
}

基本上发送数据包中的每一列，并在收到数据包后立即将其写入数据数组。 当渲染框架时，请抓取数据副本。