How to validate whether a marked feature is correctly tracking an object in a video by OpenCV?

Question

I want to validate the object detection(the green boxes) which I have marked, whether it's tracking that object only in a video.

How do I check whether it's tracking that object only and not moved to any other object? say in this case How do I validate that the left car(black) is tracking correctly over the next set of frames along with another object(white car on the right side)

edit: I have tried with finding contours and extracting particularly that object (the black car in this frame) and tried to extract features out of it, but that didn't work.

Answer 1

If you initialize your tracking with a detection step, you could periodically reiterate that detection inside the region you are tracking to make sure the car is still there. Alternatively, you could describe the object region using various histograms (color, gradients, etc.) and check whether the region you are tracking is still similar to what it started with.

I suggest checking out color histograms and HOGs (histogram of oriented gradients) to start with, and maybe combine them.

You could also analyze the objects' motion to detect irregularities, jumps, etc. Consider comparing next-frame positions to predicted positions with a Kalman filter.