I'm trying to use opencv.js to find a document in a provided image (detect edges, apply perspective transform, etc.
I've got a reasonable set of code that (occasionally) detects edges of a document and grabs the bounding box for that. However, I'm struggling to do the perspective transform steps. There are some helpers for this (not in JS) here and here .
Unfortunately I'm getting stuck on something simple. I can find the matching Mat
that has 4 edges. Displaying that shows it to be accurate. However, I have no idea how to get some simple X/Y info out of that Mat
. I thought minMaxLoc()
would be a good option, but I keep getting an error passing in my matching Mat
. Any idea why I can draw foundContour
and get bounding box info from it, but I can't call minMaxLoc
on it?
Code:
//<Get Image>
//<Convert to Gray, do GaussianBlur, and do Canny edge detection>
let contours = new cv.MatVector();
cv.findContours(matDestEdged, contours, hierarchy, cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE);
//<Sort resulting contours by area to get largest>
let foundContour = null;
for (let sortableContour of sortableContours) {
let peri = cv.arcLength(sortableContour.contour, true);
let approx = new cv.Mat();
cv.approxPolyDP(sortableContour.contour, approx, 0.1 * peri, true);
if (approx.rows == 4) {
console.log('found it');
foundContour = approx
break;
}
else {
approx.delete();
}
}
//<Draw foundContour and a bounding box to ensure it's accurate>
//TODO: Do a perspective transform
let result = cv.minMaxLoc(foundContour);
The last line above results in a runtime error ( Uncaught (in promise): 6402256 - Exception catching is disabled
). I can run minMaxLoc()
on other Mat
objects.
For anyone else looking to do this in OpenCV.JS, what I commented above seems to still be accurate. The contour found can't be used with minMaxLoc
, but the X/Y data can be pulled out of data32S[]
. That should be all that's needed to do this perspective transform. Some code is below.
//Find all contours
let contours = new cv.MatVector();
let hierarchy = new cv.Mat();
cv.findContours(matDest, contours, hierarchy, cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE);
//Get area for all contours so we can find the biggest
let sortableContours: SortableContour[] = [];
for (let i = 0; i < contours.size(); i++) {
let cnt = contours.get(i);
let area = cv.contourArea(cnt, false);
let perim = cv.arcLength(cnt, false);
sortableContours.push(new SortableContour({ areaSize: area, perimiterSize: perim, contour: cnt }));
}
//Sort 'em
sortableContours = sortableContours.sort((item1, item2) => { return (item1.areaSize > item2.areaSize) ? -1 : (item1.areaSize < item2.areaSize) ? 1 : 0; }).slice(0, 5);
//Ensure the top area contour has 4 corners (NOTE: This is not a perfect science and likely needs more attention)
let approx = new cv.Mat();
cv.approxPolyDP(sortableContours[0].contour, approx, .05 * sortableContours[0].perimiterSize, true);
if (approx.rows == 4) {
console.log('Found a 4-corner approx');
foundContour = approx;
}
else{
console.log('No 4-corner large contour!');
return;
}
//Find the corners
//foundCountour has 2 channels (seemingly x/y), has a depth of 4, and a type of 12. Seems to show it's a CV_32S "type", so the valid data is in data32S??
let corner1 = new cv.Point(foundContour.data32S[0], foundContour.data32S[1]);
let corner2 = new cv.Point(foundContour.data32S[2], foundContour.data32S[3]);
let corner3 = new cv.Point(foundContour.data32S[4], foundContour.data32S[5]);
let corner4 = new cv.Point(foundContour.data32S[6], foundContour.data32S[7]);
//Order the corners
let cornerArray = [{ corner: corner1 }, { corner: corner2 }, { corner: corner3 }, { corner: corner4 }];
//Sort by Y position (to get top-down)
cornerArray.sort((item1, item2) => { return (item1.corner.y < item2.corner.y) ? -1 : (item1.corner.y > item2.corner.y) ? 1 : 0; }).slice(0, 5);
//Determine left/right based on x position of top and bottom 2
let tl = cornerArray[0].corner.x < cornerArray[1].corner.x ? cornerArray[0] : cornerArray[1];
let tr = cornerArray[0].corner.x > cornerArray[1].corner.x ? cornerArray[0] : cornerArray[1];
let bl = cornerArray[2].corner.x < cornerArray[3].corner.x ? cornerArray[2] : cornerArray[3];
let br = cornerArray[2].corner.x > cornerArray[3].corner.x ? cornerArray[2] : cornerArray[3];
//Calculate the max width/height
let widthBottom = Math.hypot(br.corner.x - bl.corner.x, br.corner.y - bl.corner.y);
let widthTop = Math.hypot(tr.corner.x - tl.corner.x, tr.corner.y - tl.corner.y);
let theWidth = (widthBottom > widthTop) ? widthBottom : widthTop;
let heightRight = Math.hypot(tr.corner.x - br.corner.x, tr.corner.y - br.corner.y);
let heightLeft = Math.hypot(tl.corner.x - bl.corner.x, tr.corner.y - bl.corner.y);
let theHeight = (heightRight > heightLeft) ? heightRight : heightLeft;
//Transform!
let finalDestCoords = cv.matFromArray(4, 1, cv.CV_32FC2, [0, 0, theWidth - 1, 0, theWidth - 1, theHeight - 1, 0, theHeight - 1]); //
let srcCoords = cv.matFromArray(4, 1, cv.CV_32FC2, [tl.corner.x, tl.corner.y, tr.corner.x, tr.corner.y, br.corner.x, br.corner.y, bl.corner.x, bl.corner.y]);
let dsize = new cv.Size(theWidth, theHeight);
let M = cv.getPerspectiveTransform(srcCoords, finalDestCoords)
cv.warpPerspective(matDestTransformed, finalDest, M, dsize, cv.INTER_LINEAR, cv.BORDER_CONSTANT, new cv.Scalar());
For reference, here is the class definition I was using for SortableContour
. The code above is meant as a guide, not as something that can run on its own, however.
export class SortableContour {
perimiterSize: number;
areaSize: number;
contour: any;
constructor(fields: Partial<SortableContour>) {
Object.assign(this, fields);
}
}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.