Overlaying image with OpenCV4Android

Question

I've been trying to overlay an image over a rectangle detected by the camera. I'm using a function that is returning the path of my image from my sd card (.jpg), but it looks that it is not loading it correctly, or at least I can't display it where I want.

First of all, I check if the image is loaded (and I load it if it isn't). Then, I proceed recognizing rectangles. Once I recognize one, I'm suppose to overlay that rectangle on the frame with the loaded image, and then return to the ImageView the modified frame.

I'll share with you my code from the onCameraFrame function ("image" is a Mat object where I store my loaded image from my sdcard, and "img_path" is the string where the image's path is stored):

Mat dst = inputFrame.rgba();

//Here I check if the image is loaded
        if (image.empty()){
            Log.v("Image","Empty!");
            image = imread(img_path, CV_LOAD_IMAGE_UNCHANGED);
        } else {

            //If the image is loaded, then I proceed to process the frame.
            Mat gray = inputFrame.gray();


            pyrDown(gray, dsIMG, new Size(gray.cols() / 2, gray.rows() / 2));
            Imgproc.pyrUp(dsIMG, usIMG, gray.size());

            Imgproc.Canny(usIMG, bwIMG, 0, threshold);

            Imgproc.dilate(bwIMG, bwIMG, new Mat(), new Point(-1, 1), 1);

            List<MatOfPoint> contours = new ArrayList<MatOfPoint>();

            cIMG = bwIMG.clone();

            Imgproc.findContours(cIMG, contours, hovIMG, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);


            for (MatOfPoint cnt : contours) {

                MatOfPoint2f curve = new MatOfPoint2f(cnt.toArray());

                Imgproc.approxPolyDP(curve, approxCurve, 0.02 * Imgproc.arcLength(curve, true), true);

                int numberVertices = (int) approxCurve.total();

                double contourArea = Imgproc.contourArea(cnt);

                if (Math.abs(contourArea) < 100) {
                    continue;
                }

                //Rectangle detected
                if (numberVertices >= 4 && numberVertices <= 6) {

                    List<Double> cos = new ArrayList<>();

                    for (int j = 2; j < numberVertices + 1; j++) {
                        cos.add(angle(approxCurve.toArray()[j % numberVertices], approxCurve.toArray()[j - 2], approxCurve.toArray()[j - 1]));
                    }

                    Collections.sort(cos);

                    double mincos = cos.get(0);
                    double maxcos = cos.get(cos.size() - 1);

                    if (numberVertices == 4 && mincos >= -0.3 && maxcos <= 0.5) {
                        Rect r = Imgproc.boundingRect(cnt);
                        image.copyTo(dst.submat(r));

                        Log.v("Test 1",Integer.toString(image.width()));
                        Log.v("Test 2",Integer.toString(image.height()));

                    }
                }
            }
        }
        return dst;

I want to see over the rectangles the image, but I keep looking at the same unmodified. I am getting only once the log "Empty!", so it should be reading correctly the image and storing it on my Mat. Also, on the other 2 Logs where I am writting the image number of rows and columns, I am getting a number bigger than 0. So...

Is that the right way of checking if the image is correctly loaded? Do you now why that code is not working? I also read about the function addWeighted, but I tried to resized the smaller image and it failed.

Thanks in advance!!

EDIT This is the part of the code where I recognize the rectangle and I have to render the new image over the rectangle on the frame:

 //Rectangle detected
                if (numberVertices >= 4 && numberVertices <= 6) {

                    List<Double> cos = new ArrayList<>();

                    for (int j = 2; j < numberVertices + 1; j++) {
                        cos.add(angle(approxCurve.toArray()[j % numberVertices], approxCurve.toArray()[j - 2], approxCurve.toArray()[j - 1]));
                    }

                    Collections.sort(cos);

                    double mincos = cos.get(0);
                    double maxcos = cos.get(cos.size() - 1);

                    if (numberVertices == 4 && mincos >= -0.3 && maxcos <= 0.5) {
                        Rect r = Imgproc.boundingRect(cnt);
                        image.copyTo(dst.submat(r));

                        Log.v("Test 1",Integer.toString(image.width()));
                        Log.v("Test 2",Integer.toString(image.height()));

                    }
                }

As an example, you can check the following image from the InkHunter app's webpage: https://i1.wp.com/goosed.ie/wp-content/uploads/2016/10/inkhunter-cover-for-tattoo-ideas-1.jpg?fit=800%2C450&ssl=1

Answer 1

I found the solution.

Both image size and image channels number must be same, and use mask for faster processing images.