OpenCV: How to detect (and draw rectange on it) all moving objects at video of moving camera?

Question

I want to find all moving objects at video. Right now, I find a foreground mask using background subtraction, then I find contours on that mask and then draw rectangles around contours.

But there is possible situation when some interesting object moves to the border on frame and I want to rotate camera to keep it in center. In that case the background moves faster than object and tracker starts tracking large background objects instead of moving object.

For ie we tracking ball in the football game (and there also can be a bold heads :) ) and camera rotates when ball moves to another half of field.

Is it possible to compensate the camera movement to prevent moving object tracking fail?

Is there better way to detect all moving objects on video from moving camera?

Answer 1

If you know precisely by how much the background has moved, you can compensate by shifting one of the images before subtraction.

Anyway, this will not work perfectly because the transformation is not a pure shift, some deformation is added. If you have a model of lens distortion, you can compensate, but this is a relatively sophisticated operation.

Last but not least, if the camera does not rotate around its optical axis or translates, you can't exactly correct the distortion, which depends on (unknown) depth.