视频转换：h.264从UDP套接字到Mat对象

Question

I have a Raspberry pi with a picam and I wanto to stream video to a server to process it and run object detection software. Im sending a video stream over a UDP socket to the server and reciving byte arrays that I can't turn into Mats.

The python sender:

import socket
import time
import picamera
import sys

# msg = b'test'
addr = ("192.168.1.79", 7777)
start = time.time()
# Connect a client socket to my_server:8000 (change my_server to the
# hostname of your server)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(('192.168.1.79', 7777))
con = s.makefile('wb');
print("Conectado")
# Make a file-like object out of the connection
# connection = s.makefile('wb')
try:
    with picamera.PiCamera() as camera:
        camera.resolution = (640, 480)
        camera.framerate = 24
        print("Cargando camara")
        camera.start_preview()
        time.sleep(2)
        print("Enviando")
        camera.start_recording(con, format='h264')
        camera.wait_recording(10)
        camera.stop_recording()        
finally:
    con.close()
    s.close()

The c# reciver:

using Emgu.CV;
using Emgu.CV.UI;
using System;
using System.Drawing;
using System.Net;
using System.Net.Sockets;


namespace EgmuCv {
    class Decoder{
        private const int listenPort = 7777;


        public static void streamToImage(ImageBox img) {
            UdpClient listener = new UdpClient(listenPort);
            IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, listenPort);
            Mat mat = new Mat(new Size(640, 480), Emgu.CV.CvEnum.DepthType.Cv8U, 3);

            try {
                while (true) {
                    Console.WriteLine("Waiting for broadcast");
                    byte[] bytes = listener.Receive(ref groupEP);
                    // CvInvoke.Imdecode(bytes, Emgu.CV.CvEnum.ImreadModes.ReducedColor8, mat);
                    /** Code to turn bytes into Mat **/
                    // img.Image = mat;
                }
            } catch (SocketException e) {
                Console.WriteLine(e);
            } finally {
                listener.Close();
            }

        }
    }
}

Answer 1

After some research I found thath h.264 shold no be streamed trought UDP but TCP, and h.264 is not a good idea at all if you want to use the video in a Object Detection program. A better idea is to send individual MJPEG frames with a TCP socket, is a little bit slower but im geting 640X480@30 which is more than enouht and the conversion to a Mat object way easyer