OpenCV Isolate silhouette from webcam stream

Question

I'm trying to isolate the silhouette of a person from an unknown video stream. ( The users webcam ), using C++/Cinder/OpenCV . I've got as far as identifying & drawing contours, but I'm not getting a contour of the whole person, just elements ( hair, eyes, etc. )

I'm using: BackgroundSubtractorMOG2 to remove the background. Blur to remove noise. Adaptive threshold. Find & draw contours over a certain complexity.

Code:

Surface surface;
surface = mCapture.getSurface();

// To texture for display
textureCapture = Texture( surface );

    // Greyscale
    Mat matGrey( toOcv( surface ) );

    // Output
    Mat matForeground, matBackground;

    // Build foreground & background
    mog( matGrey, matForeground, -1 );
    mog.getBackgroundImage( matBackground );

// Build countours
Mat matContourTemp = matForeground.clone();

    // Blur to remove noise
    Mat matBlurred = matContourTemp.clone();
    cv::GaussianBlur( matContourTemp, matBlurred, cv::Size( 9, 9 ), 0 );

    textureBlurred = Texture( fromOcv( matBlurred ) );

    // Adaptive threshold
    Mat matThresh = matContourTemp.clone();
    adaptiveThreshold( matBlurred, matThresh, 255, CV_ADAPTIVE_THRESH_MEAN_C, CV_THRESH_BINARY, 3, 0 );

    textureThreshold = Texture( fromOcv( matThresh ) );

    // Contours
    vector<cv::Vec4i> hierarchy;

        // Find
        contours.clear();
        findContours( matThresh, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE, cv::Point( 0, 0 ) );

        // Draw contours
        Mat matContourImage( matForeground.size(), CV_8UC3, cv::Scalar( 0, 0, 0 ) );

        Scalar colors[ 3 ];
        colors[ 0 ] = Scalar( 255, 255, 255 );

        for( size_t idx = 0; idx < contours.size(); idx++){

            if( contours[ idx ].size() > 40 ){

                cv::drawContours(
                        matContourImage, contours, idx,
                        colors[ 0 ], -3,
                        100,
                        hierarchy,
                        0,
                        cv::Point( 0, 0 ) );
                };
    };
    textureContour = Texture( fromOcv( matContourImage ) );

Output: ( I'm too junior here to post images )

http://barnabysheeran.com/outgoing/stackoverflow/ss_1.png http://barnabysheeran.com/outgoing/stackoverflow/ss_2.png

I'd like this to be one whole body filled silhouette.

Answer 1

You could use erode/dilate after thresholding, which in general removes noise and stretches white areas, but I suggest to use BGSlibrary I used it in past in conjunction with openFrameworks, and it is really great collection of different algorithms for subtracting background.

Also some image processing before segmentation might help a little, but that's just a theory.