OpenCV Error: Assertion failed (a_size.width == len)

Question

Background

I am currently trying to build an autonomous drone using ROS on my Rapsberry Pi which is running an Ubuntu MATE 16.04 LTS . Solving the Computer Vision problem of recognising red circles as of now. Since by nature, the drone is not stable (as there is an internal PID controller stabilising the drone) and due to lighting conditions, the drone often detects the same circle but in a very unstable way. To tackle this problem, comments on [this][1] question suggested that I try video stabilization.

Error

This is the error I am currently looking at:

OpenCV Error: Assertion failed (a_size.width == len) in gemm, file
/tmp/binarydeb/ros-kinetic-opencv3-3.1.0/modules/core/src/matmul.cpp,
line 900 terminate called after throwing an instance of
'cv::Exception'   what(): 
/tmp/binarydeb/ros-kinetic-opencv3-3.1.0/modules/core/src/matmul.cpp:900:
error: (-215) a_size.width == len in function gemm

Code

class Tracker {

    cv::Mat prevGray;
    vector<cv::Point2f> trackedFeatures;

    void processImage(const sensor_msgs::ImageConstPtr& msg) {

        // ROS declarations 
        cv_bridge::CvImagePtr cv_ptr;

        // Function Initializations
        if (freshStart == true) {
            cv::Mat rightTransform = cv::Mat::eye(3,3,CV_32FC1);
        }
        cv::Mat gray; 
        cv::Mat copy_img;
        vector<cv::Point2f> corners;

        try { 
             cv_ptr = cv_bridge::toCvCopy(msg, 
                        sensor_msgs::image_encodings::BGR8);
        } 
        catch (cv_bridge::Exception& e) {
            ROS_INFO("cv_bridge exception");
            return;
        }

        copy_img = cv_ptr->image;
        cv::cvtColor(cv_ptr->image, gray, cv::COLOR_BGR2GRAY);

        if (trackedFeatures.size() < 200) {
            cv::goodFeaturesToTrack(gray,corners,200,0.01,10);
            for (int i = 0; i < corners.size(); ++i) {
                trackedFeatures.push_back(corners[i]);
            }
        }

        if (!prevGray.empty()) {
            vector<uchar> status;
            vector<float> errors; 

            calcOpticalFlowPyrLK(prevGray, gray, trackedFeatures, corners,
                    status, errors, Size(10,10));

            if (countNonZero(status) < status.size() * 0.8) {
                cout << "cataclysmic error\n";
                rigidTransform = cv::Mat::eye(3,3,CV_32FC1);
                trackedFeatures.clear();
                prevGray.release();
                freshStart = true;
                return;
            } else {
                freshStart = false;
            }

            cv::Mat_<float> newRigidTransform = estimateRigidTransform(
                    trackedFeatures, corners, false);
            cv::Mat_<float> nrt33 = cv::Mat_<float>::eye(3,3);
            newRigidTransform.copyTo(nrt33.rowRange(0,2));
            rigidTransform *= nrt33;

            trackedFeatures.clear();
            for (int i = 0; i < status.size(); ++i) {
                if (status[i]) {
                    trackedFeatures.push_back(corners[i]);
                }
            }
        }

        // Debugging to see the tracked features as of now
        for (int i = 0; i < trackedFeatures.size(); ++i) {
            circle(cv_ptr->image, trackedFeatures[i], 3, Scalar(0,0,255), 
                CV_FILLED);
        }

        imshow(OPENCV_WINDOW, cv_ptr->image); 
        cv::waitKey(3);

        gray.copyTo(prevGray);
    }
};

Now I know the error lies in the statement gray.copyTo(prevGray) . This is due to the fact that I get no errors when I comment this out.

Overall Structure of the Program

class Tracker {

    // The ROS declarations 
    ...

    // Internal Declarations
    ...

    public:
    bool freshStart;
    Mat_<float> rigidTransform;
    ...

    Tracker():it_(nh1_) {
        image_sub_ = it_.subscribe("/ardrone/bottom/image_raw", 1,
                                        &Tracker::processImage, this);
        image_pub_ = it_.advertise("/stabImage", 1);

        cv::namedWindow(OPENCV_WINDOW);
    }

    ~Tracker() {
        cv::destroyWindow(OPENCV_WINDOW);
    }

    void processImage(const sensor_msgs::ImageConstPtr& msg) {
    ...
    }

int main(int argc, char** argv)
{
  ros::init(argc, argv, "video_stabilizer");
  Tracker tr;
  ros::spin();
  return 0;
}

Answer 1

The problem seems to be a matmul issue. Where do you initialize rigidTransform ?

I think instead of the line:

cv::Mat rightTransform = cv::Mat::eye(3,3,CV_32FC1);

you need:

rigidTransform = cv::Mat::eye(3,3,CV_32FC1);

The problem does not happen when you comment out gray.copyTo(prevGray) because without it the loop if if (!prevGray.empty()) does not run.