Convert canvas pixel information (ImageData object) into MarvinImage for further processing

Question

I have a small application that allows the user to draw a shape and then processes the image created in canvas as a result of the user's input. The goal is character recognition. For instance, you draw the letter 'g', then the app suggests letters from a list that match the shape you've just drawn sorted by most similar (hopefully 'g' is right at the top).

For this, I need to do some image processing on the image data taken from the canvas. Every JS image processing library I've found is designed to use a loaded image. For instance, MarvinJ is easily able to load an image from URL, and every example I've found does this. The docs describe the possibility of creating a new image from an object, but I cannot get this to work with the ImageData object returned by:

let image = ctx.getImageData(0, 0, canvas.width, canvas.height);

When I try to pass the ImageData object to the MarvinJ constructor directly:

let imageEdge = new MarvinImage(image.data);

I get the following error:

[Error] TypeError: Value NaN is outside the range [-2147483648, 2147483647] getImageData (marvinj-0.9.js:152) (anonymous function) (marvinj-0.9.js:152) MarvinImage (marvinj-0.9.js:136) runSearch (search.js:97) up (search.js:66)

So ImageData is not suited for creating a new MarvinImage. I can't seem to figure out how to make it suitable. It's just an Array of integer values, so I don't even know what the difficulty is.

Minor note: I don't particularly care which image processing library I use. Any decent one will do. I just don't want to write my own edge finder. No sense reinventing the wheel. But the problem is that I can't find any library that deals with the situation of taking the source image from canvas rather than from a file.

 let canvas = document.getElementById("signPad"); let ctx = canvas.getContext("2d"); ctx.fillStyle = "#FFFFFF"; ctx.fillRect(0,0,canvas.width, canvas.height); ctx.strokeStyle = "#000000"; ctx.lineWidth = 18; const rect = canvas.getBoundingClientRect(); let image = ctx.getImageData(0, 0, canvas.width, canvas.height); const getCursorPosition = (canvas, event) => { let x, y; // For touch events if (event.touches) { x = event.touches[0].clientX - rect.left; y = event.touches[0].clientY - rect.top; } else { x = event.clientX - rect.left; y = event.clientY - rect.top; } return {x: x, y: y}; }; let mouseDown = false; const down = e => { mouseDown = true; ctx.beginPath(); e.preventDefault(); }; const up = e => { mouseDown = false; console.log('Run search'); runSearch(); }; const move = e => { let coords = getCursorPosition(canvas, e); if (mouseDown) { ctx.lineTo(coords.x, coords.y); ctx.stroke(); image = ctx.getImageData(0, 0, canvas.width, canvas.height); } e.preventDefault(); }; canvas.addEventListener('mousedown', down); canvas.addEventListener('touchstart', down); canvas.addEventListener('mousemove', move); canvas.addEventListener('touchmove', move); canvas.addEventListener('mouseup', up); canvas.addEventListener('touchend', up); const clearPad = e => { ctx.clearRect(0, 0, canvas.width, canvas.height); } console.log(image); const runSearch = () => { // let imageEdge = new MarvinImage(image.data); }

 <canvas id="signPad" width="400" height="400" > </canvas> <div> <button onClick="clearPad()">Clear</button> </div> <script src="https://www.marvinj.org/releases/marvinj-0.9.js"></script>

Answer 1

Looking through MarvinJ's sourcecode on Github , the constructor for MarvinImage does not accept an imagedata object. The only valid parameters are width , height and a colorModel .

To send an image to MarvinJ you need to use the load() method of the MarvinImage class. The URL you send to load does not have to be an internet address however. If you have a html <canvas> element, you can call the .toDataURL() on it to get a base64 .png representation of the canvas' content. This data can be used by MarvinJ.

Here's an example:

 let canvas = document.getElementById("canvas"); let ctx = canvas.getContext("2d"); let marvinImage; let output; for (let a = 0; a < 100; a++) { ctx.beginPath(); ctx.strokeStyle = "#000000".replace(/0/g, () => { return (~~(Math.random() * 16)).toString(16); }) ctx.arc(Math.random() * canvas.width, Math.random() * canvas.height, Math.random() * 20, 0, 2 * Math.PI); ctx.stroke(); ctx.closePath(); } document.getElementById("gen").onclick = (e) => { marvinImage = new MarvinImage(); marvinImage.load(canvas.toDataURL(), myCallback); } function myCallback() { output = new MarvinImage(marvinImage.getWidth(), marvinImage.getHeight()); Marvin.blackAndWhite(marvinImage, output, 100); output.draw(document.getElementById("filtered")); }

 <script src="https://www.marvinj.org/releases/marvinj-0.9.js"></script> <canvas id="canvas" width="150" height="150"></canvas> <button type="button" id="gen">Filter</button> <canvas id="filtered" width="150" height="150"></canvas>