tracking multiple objects using calcOpticalFlowPyrLK

Question

The below code of snippet is for tracking but the problem is i don't know why for some reasons the points(centroid points of object) disappears for few frames parallely calcOpticalFlowPyrLK is also not working for those frames everything disappears for few frames. Images are as shown below. Can anyone Please help to track these objects continuously. I also tried to do with kalman filter but couldn't succeed.

Frame 10 of the video

Frame 25 of the video

Look at this image points disappears just an input image is displayed

Frame 45 of the video

    for(int i =0; i<theVehicles.size(); i++){
    Point p=Point(theVehicles.at(i).getXPos(),theVehicles.at(i).getYPos());
    points12.push_back(p);
    TermCriteria termcrit(TermCriteria::COUNT|TermCriteria::EPS,20,0.03);
    Size subPixWinSize(10,10), winSize(31,31);

    //cout<<p;

   calcOpticalFlowPyrLK(prevGray, filledEdgesOut,  points12,points, status, err, winSize, 3, termcrit, 0, 0.001);

   RNG rng(12345);

            for( int i = 0; i < points.size(); i++ )
     { 
         circle( frame, points[i], r, Scalar(rng.uniform(0,255), rng.uniform(0,255),rng.uniform(0,255)), -1, 8, 0 ); 

         /*cv::circle(frame,points[i],10,cv::Scalar(0,0,255));
         cv::putText(frame,intToString(theVehicles.at(i).getXPos())+ " , " + intToString(theVehicles.at(i).getYPos()),cv::Point(theVehicles.at(i).getXPos(),theVehicles.at(i).getYPos()+20),1,1,Scalar(0,255,0));
    cv::putText(frame,theVehicles.at(i).getType(),cv::Point(theVehicles.at(i).getXPos(),theVehicles.at(i).getYPos()-30),1,2,theVehicles.at(i).getColour());*/
            }

    }

Answer 1

If you are doing this in a loop (for each frame for example) then you have to re-generate the points on the object once it's detected and tracked correctly. For example, if you are using a bounding-box to describe your vehicles then once you match a vehicle between two consecutive frames you have to generate a new set of points by using good features or uniformly on the new detected vehicle.