Q: Single object tracking in python using opencv

Question

What is the best way to track mario in this video ?

I tried many algorithms like CAMShift .. and tried Object Tracking Algorithms in opencv 3 like BOOSTING, MIL, TLD, MEDIANFLOW but all of them failed with super mario

you can see the video from here : super mario video Link

Answer 1

It seems Supermario is programmed in a way that the whole environement (background) moves, if Mario is at the center of the frame. I suggest to track Mario this way:

Lets asume img[t1] and img[t2] are two successive frames in the video

• Check if background moves (diff = img[t1] - img[t2])
  • if diff > 0 for (almost) all pixels (background moves): Mario must be in the center of the image
  • if diff == 0 for (almost) all pixels: Mario can be found with Background subtraction (explained here )

With background subtraction also other moving objects (like fire balls or turtles) will be found. However it should be rather simple to find mario in this limited search space.

There is one case left that is not covered here: Mario does not move, and is infront of the center. here the position of Mario at t1 and t2 is unchanged.