Calculate CV2 Homography by points and line

Question

I have a list of points in the field (like upper_goal_point / left_upper_outer_corner , etc.

I know their corresponding coordinates in destination image - so I can calculate the homography:

h, status = cv2.findHomography(pts_src, pts_dst)

I also have blue points in the upper corner line (look at image above), which I only know that their destination's y coordinates are 0 (because they are in the upper line), but I don't know where exactly they lay in that line.

Can I use those blue points in order to improve the homography?

PS

You can see that the upper corner line in the homography is not horizontal line, it's diagonal, which of course is not correct:

Answer 1

Actually it possible to use line correspondence in find homography.

https://www.researchgate.net/publication/220845575_Combining_Line_and_Point_Correspondences_for_Homography_Estimation

Several years ago we implement this approach in one project. During simplification all math we come up with simple trick. We transform every line a*x + b*y + c = 0 to point (a/c, b/c)

// ***  Don't copy paste this code, read below!  ***//
Point2f convertPointsToHomogeneousLine(Point2f p1, Point2f p2) {
    Point3f p1h(p1.x, p1.y, 1);
    Point3f p2h(p2.x, p2.y, 1);
    Point3f lineHomo(p1h.y*p2h.z - p1h.z*p2h.y,
                     p1h.z*p2h.x - p1h.x*p2h.z,
                     p1h.x*p2h.y - p1h.y*p2h.x);
    
    Point2f lineHomoNorm(lineHomo.x / lineHomo.z,
                         lineHomo.y / lineHomo.z);
    return lineHomoNorm;
}

And pass this points inside. As I remember I also dig inside OpenCV implementation of findHomography and insert this lines somewhere inside to solve step. Inside OpenCV there some normalization applied to points before pass to solve step. So you need to skip this step for this kind of points.

We do not use it in production. User need to calibrate camera manually by providing lines and points on the image and in meter system. It has too complicated interface and bad stability. But in your case I think it can work better. If you will automatically find lines and correspondence.

PS Please note that in paper they use some normalization technique. It will improve stability. We faced with stability problem, do not solved it in our journey.