通过TCP将JPEG从iPhone发送到PC

Question

How to prevent a crash when JPEG image data sent from a client doesn't match the size read on a server due to an incomplete image?

1. Client: Send 4 bytes of image length info-------->Server: Read 4 bytes.
2. Client: Send x bytes of image data --------------->Server: Read x bytes.

I use sampleBuffer (AVFoundation) to create JPEG images on iPhone, send the length of each image before sending the image data. Images are sent continuously at a time interval. I use Microsoft Vsual Studio 2005(C++) to receive the data and display the image using OpenCV.

iPhone (Client):

// AVCaptureSession delegate
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection 
{ 

    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer,0); 

    uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer); 
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
    size_t width = CVPixelBufferGetWidth(imageBuffer); 
    size_t height = CVPixelBufferGetHeight(imageBuffer);  

    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 
    CGContextRef newContext = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
    CGImageRef newImage = CGBitmapContextCreateImage(newContext); 

    CGContextRelease(newContext); 
    CGColorSpaceRelease(colorSpace);

    [self.customLayer performSelectorOnMainThread:@selector(setContents:) withObject: (id) newImage waitUntilDone:YES];

    UIImage *image= [UIImage imageWithCGImage:newImage scale:1.0 orientation:UIImageOrientationRight];
    NSData *data = UIImageJPEGRepresentation(image, 0.5);

    // Send image length bytes
    uint32_t length = (uint32_t)htonl([data length]);
    [_outputStream write:(uint8_t *)&length maxLength:4];

    // Send image data bytes
    [_outputStream write:(const uint8_t *)[data bytes] maxLength:[data length]];

    CGImageRelease(newImage);
    CVPixelBufferUnlockBaseAddress(imageBuffer,0);
    [pool drain];
}

PC - VS2005 C++ (Server):

cvNamedWindow("image", CV_WINDOW_AUTOSIZE);

std::vector<char> buf;

for(int i=0; i < 1000; i++) {

    typedef unsigned __int32 uint32_t;
    uint32_t len;
    int lengthbytes = 0;
    int databytes = 0; 

    // Receive image length bytes
    lengthbytes = recv(clientSocket, (char *)&len, sizeof(len), 0); 

    len = ntohl(len);

    buf.clear();
    buf.resize(len);

    // Receive image data bytes ()
    databytes = recv(clientSocket, &buf[0], buf.size(), 0);

    // Display image
    Mat matrixJpeg = imdecode(Mat(buf), 1);
    IplImage object = matrixJpeg;
    IplImage* fIplImageHeader = &object;
    cvShowImage("image", fIplImageHeader);
    cvWaitKey(50);
}

Answer 1

Here is the solution of my problem:

cvNamedWindow("image", CV_WINDOW_AUTOSIZE);

std::vector<char> buf;

for(int i=0; i < 1000; i++) {

    typedef unsigned __int32 uint32_t;
    uint32_t len;
    int lengthbytes = 0;
    int databytes = 0; 

    // Receive image length bytes
    lengthbytes = recv(clientSocket, (char *)&len, sizeof(len), 0);

    len = ntohl(len);

    buf.clear();
    buf.resize(len);

    // Receive image data bytes      
    databytes = recv(clientSocket, &buf[0], buf.size(), 0);

    if ( buf[0] == (char)0xff && buf[1] == (char)0xd8 && buf[buf.size() - 2] == (char)0xff && buf[buf.size() -1] == (char)0xd9) {

        // Display image if it's a complete JPEG file with SOF = FF D8 and EOF = FF D9
        Mat matrixJpeg = imdecode(Mat(buf), 1);
        IplImage object = matrixJpeg;
        IplImage* fIplImageHeader = &object;
        cvShowImage("image", fIplImageHeader);
        cvWaitKey(50);

    }else{

        // Clear buffer and move to the SOF of the next image data
        buf.clear();
        buf.resize(2);

        while(recv(clientSocket, &buf[0], 1, 0) == 1){     
            if(buf[0] == (char)0xff) { 
                databytes1 = recv(clientSocket, &buf[1], 1, 0);
                if(buf[1] == (char)0xd9 ) { 
                    buf.clear();
                }
            }
        }
    }
}