opencv video stabilization algorithm

Question

I am writing video stabilizer using opencv. The algorithm is as follows:

while there are more frames in the video:

1. take new frame from the video
2. detect keypoints in the new frame
3. compute descriptor for new keypoints
4. match descriptors of the new and the previous frame
5. filter matches to get good matches
6. find homography between previous and new frame
7. apply homography (warpPerspective) to the new frame and thus create "adjusted new frame"
8. set previous frame to be equal to "adjusted new frame" (descriptors, keypoints)

I have a few questions. Am I on the right track? How to do the actual stabilization (using Gaussian filter or something else)?

Answer 1

Here is possible sequence of steps:

Step 1. Read Frames from a Movie File

Step 2. Collect Salient Points from Each Frame

Step 3. Select Correspondences Between Points

Step 4. Estimating Transform from Noisy Correspondences

Step 5. Transform Approximation and Smoothing

Step 6. Run on the Full Video

More details on each step you can find here:

http://www.mathworks.com/help/vision/examples/video-stabilization-using-point-feature-matching.html

I think you can follow the same steps in OpenCV.

Answer 2

If you're using python code then you can use my powerful & threaded VidGear Video Processing python library that now provides real-time Video Stabilization with minimalistic latency and at the expense of little to no additional computational power requirement with Stabilizer Class . Here's a basic usage example for your convenience:

# import libraries
from vidgear.gears import VideoGear
from vidgear.gears import WriteGear
import cv2

stream = VideoGear(source=0, stabilize = True).start() # To open any valid video stream(for e.g device at 0 index)

# infinite loop
while True:

    frame = stream.read()
    # read stabilized frames

    # check if frame is None
    if frame is None:
        #if True break the infinite loop
        break

    # do something with stabilized frame here

    cv2.imshow("Stabilized Frame", frame)
    # Show output window

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

cv2.destroyAllWindows()
# close output window

stream.stop()
# safely close video stream

More advanced usage can be found here: https://github.com/abhiTronix/vidgear/wiki/Real-time-Video-Stabilization#real-time-video-stabilization-with-vidgear