Object Tracking Using Open CV

Question

I am planning to do a project which involves tracking various kinds of objects (including people, vehicles, etc) from the video input of a CCTV camera. I am intending to use OpenCV. The implementation should be tolerant to occlusion and should be able to recognize and continue tracking objects even if they exit the frame and re-enter after a while. Also new entries into the frame should be identified automatically and tracked and I need to extract the positions of the objects over time. Since there are a lot of options available, I am actually confused. Can somebody suggest the best way forward. I have already tried the following:

• I tried using the Tracking API of Open CV and tried out all of the following algorithms: MIL, TLD, BOOSTING, MEDIANFLOW. But none of them seem to be accurate enough.
• I also know about the Kalman Filter. But it cannot handle the problem of recognizing a previous object that has re-entered the frame from a different position. I would prefer a solution that incorporates this feature too.
• I tried out Camshift. But it requires specification of intensity ranges. Since the actual setup of the CCTV installation is not known before-hand, this is not an option either.
• I also looked up the HOGDescriptor class. But it does not provide a facility for tracking directly right? I mean only detection is performed.

Are there better solutions that are more suitable to the problem at hand than the ones mentioned above? Are there any ways to improve the accuracy of the Tracking algorithms? And are there ways to improve on the solutions that I have already considered? Please help to solve my confusion.

Answer 1

You are basically looking for the perfect tracking algorithm :) If you want to know which fits best for your requirements, you can check the benchmarks such as Visual Object Tracking Challenge , Visual Tracker Benchmark . It may be tough to implement though.

On the other hand, I guess that you want real time performance. In this case, you should check my this answer, too.

I also know about the Kalman Filter. But it cannot handle the problem of recognizing a previous object that has re-entered the frame from a different position. I would prefer a solution that incorporates this feature too.

There is no such prediction algorithm. You have to check every possible border position to locate the target again.

I tried out Camshift. But it requires specification of intensity ranges. Since the actual setup of the CCTV installation is not known before-hand, this is not an option either.

Since you practically want the best algorithm ever, probably you wouldn't prefer camshift. A histogram based algorithm is not robust to similar targets (or any object with similar colors, there are some algorithms which is histogram based and try to deal with shapes ( fragTrack is a good example). Yet, i think it won't satisfy you.)

I also looked up the HOGDescriptor class. But it does not provide a facility for tracking directly right? I mean only detection is performed.

Practically, to detect a target you need to train it first (or (for example) you could use an objectness algorithm to detect salient objects in the scene). Which means you have to train it for every possible target. So, if you detect one of the targets trained before, you could try to detect the same target in every frame. You could use a Kalman filter to narrow down the search gate and get better results.

To sum up, start with benchmarks. Find the best algorithm that fits for you.

By the way, deep learning is the trend for computer vision algorithms nowadays. Probably the best one comprises deep learning one way or another.

Hope this helps,
Gokhan.