使用OpenCV跟踪对象

Question

I have a video clip with fish swimming in a flume and small particles floating on the surface. So, the moving objects are the swimming fish and hundreds of particles. Here is the link of the video sample .

My interest is to locate the fish correctly in each single frame.

Currently, my procedures are as below:

1. remove light reflection (moving noise) in each single frame by converting RGB to HSV color space.
2. apply background subtraction method cv2.BackgroundSubtractorMOG .
3. use erosion and dilation to remove most of the moving particle noise.
4. find all contours of the blobs in binary images, and compute rotated bounding boxes for all blobs.
5. compute aspect ratios for all rotated bounding boxes.
6. identify fish in the foreground frames by using simple criteria: if a blob occupies more than 1 pixel, then check the aspect ratio of its rotated bounding box. If it has the largest aspect ratio, it is considered as fish.

My questions are:

1. In the first 10 frames, the moving fish is not identified by the cv2.BackgroundSubtractorMOG , I adjusted history , nmixture in the function, it does not seems to work well in the first 10 frames. How to pick up the moving objects in the first few frames?
2. In some frames, the cv2.BackgroundSubtractorMOG can pick up a vague outline of fish, as shown in the red circle in the pic below: Is there any method to fill up that spotted outline and identify it as a fish?

Here is the link of my code .

Thank you!

Answer 1

Assuming that the fish swim along only this direction (or the opposite), you can try to use custom morphological operations that help to grow elongated objects. It should at least increase the size of the blob, thereby giving more data to the MoG model for tracking.