在Stitcher类中将composePanorama与OpenCV-Python绑定一起使用

Question

I'm trying to estimate transform for some images and stitch them using stitcher.estimateTransform() and stitcher.composePanorama() in python. After estimating transform, composePanorama gives the error as below:

pano is not a numpy array, neither a scalar

I tried to convert NumPy Array into a Mat object using cv2.fromarray(left) , but it only works for cv, not cv2. Therefore how can I convert this numpy to MAT array in cv2. I don't find any examples of using composePanorama with python bindings. Any solution for this error or example of using stitcher.estimateTransform() with OpenCV-Python bindings would be appreciated.

Note : Although Stitching class in OpenCV-Python bindings is not complete (beacuse of auto-generated bindings), help(cv2.createStitcher()) demonstrates that it contains composePanorama() and estimateTransform() .

Note : I can use stitcher.stitch() without any problems, but using stitcher.stitch() does not help me, because I'm trying to not calculate the transform for each iteration in the main loop.

My simple code :

leftStream = cv2.VideoCapture(0)
rightStream = cv2.VideoCapture(1)
left = leftStream.read()[1]
right = rightStream.read()[1]
st = cv2.createStitcher(False)
st.estimateTransform([left, right])
st.composePanorama([left, right])

Answer 1

I have the same problem. From what I can see, composePanorama has two overloads.

CV_WRAP Status composePanorama(OutputArray pano);
Status composePanorama(InputArrayOfArrays images, OutputArray pano);

It's the second overload that we need, as the pano is an output parameter, which in Python is given as a return value. Unfortunately, the second overload is not marked by CV_WRAP which would make it available to the Python bindings. So the only solutions I can see are:

• Use an alternative stitching implementation
• Go through the C++ code of the missing composePanorama implementation and reimplement it yourself in Python
• Register an issue on the Open CV Github and wait for an update
• Build Open CV yourself from source and mark the function as CV_WRAP (I'm not sure it is actually as simple as that)
• Work in C++ instead of Python

Although I'll be very happy if someone else can post an answer showing how to achieve this in Python without going through the complex tasks above.

Answer 2

To use stitcher.estimateTransform() and stitcher.composePanorama() you will need to

1. Download opencv https://github.com/opencv/opencv
2. navigate to opencv-master/modules/stitching/include/opencv2/stitching.hpp
3. add CV_WRAP in front of any methods you want to be able to call in Python. In this case those would be estimateTransform and composePanorama

Then to build the python module:

cd ~/opencv
mkdir build
cd build
cmake ../
make
sudo make install

Then move the module to your virtual environment from wherever it was installed to. In my case that was /usr/local/lib/python3.7/site-packages/cv2.

See https://www.pyimagesearch.com/2018/08/17/install-opencv-4-on-macos/ and https://docs.opencv.org/4.1.0/da/d49/tutorial_py_bindings_basics.html and https://docs.opencv.org/4.1.1/da/df6/tutorial_py_table_of_contents_setup.html for more info.