Auto adjust brightness/contrast to read text from images

Question

I was wondering if anyone can point me in the right direction to auto adjust brightness/contrast of an image taken from phone camera using javascript to make reading of text from the image easier.

Appreciate any help,

Many thanks.

Answer 1

To automatically adjust an image we could use a histogram that we generate from the image, and then use a threshold to find a black/white point to use to scale the pixel values to their max in opposite ends.

In HTML5 we would need to use the canvas element in order to read pixel information.

Building a histogram

A histogram is an overview of which values are most represented in an image. For brightness-contrast we would be interested in the luma value (the perceived lightness of a pixel).

^{Example luma histogram}

To calculate a luma value we can use REC.709 (AKA BT.709, recommended, used here) or REC.601 formulas.

Y = 0.299 * R + 0.587 * G + 0.114 * B

We need to convert this to an integer ( iluma = Math.round(luma); ), otherwise we would get a hard time building the histogram which is based on integer values [0, 255] for storage (see example code below).

The strategy to determine which range to use can vary, but for simplicity we can choose a threshold strategy based on a minimum representation of pixels in both end.

^{Red line showing example threshold}

To find the darkest based on a threshold we would scan from left to right and when we get a luma value above threshold use that as minimum value. If we get to center (or even just 33% in) we could abort and default to 0.

For the brightest we would do the same but from right to left and defaulting to 255 if no threshold is found.

You can of course use different threshold values for each end - it's all a game of trial-and-error with the values until you find something that suits your scenario.

We should now have two values representing the min-max range:

^{Min-max range based on threshold}

Scaling the general luma level

First calculate the scale factor we need to use based on the min-max range:

scale = 255 / (max - min) * 2

We will always subtract min from each component even if that means it will clip (if < 0 set the value to 0). When subtracted we scale each component value using the scale factor. The x2 at the end is to compensate for the variations between luma and actual RGB values. Play around with this value like the others (here just an arbitrary example).

We do this for each component in each pixel (0-clip and scale):

component = max(0, component - min) * scale

When the image data is put back the contrast should be max based on the given threshold.

Tips

You don't have to use the entire image bitmap to analyze the histogram. If you deal with large image sources scale down to a small representation - you don't need much as we're after the brightest/darkest areas and not single pixels.

You can brighten and add contrast an image using blending modes with it self, such as multiply , lighten , hard-light / soft-light etc. (<= IE11 does not support blending modes). Adjust the formula for these, and just experiment.

Example

This works on a buffer showing the techniques described above. There exist more complex and accurate methods, but this is given as a proof-of-concept (licensed under CC-3.0-by-sa, attribution required ).

It starts out with a 10% threshold value. Use slider to see the difference in result using the threshold. The threshold can be calculated via other methods than the one shown here. Experiment!

Run snippet using Full page -

 var ctx = c.getContext("2d"), img = new Image; // some demo image img.crossOrigin =""; // needed for demo img.onload = setup; img.src = "//i.imgur.com/VtNwHbU.jpg"; function setup() { // set canvas size based on image c.width = this.width; c.height = this.height; // draw in image to canvas ctx.drawImage(this, 0, 0); // keep the original for comparsion and for demo org.src = c.toDataURL(); process(this, +tv.value); } function process(img, thold) { //thold = % of hist max var width = img.width, height = img.height, idata, data, i, min = -1, max = -1, // to find min-max maxH = 0, // to find scale of histogram scale, hgram = new Uint32Array(width); // histogram buffer (or use Float32) // get image data idata = ctx.getImageData(0, 0, img.width, img.height); // needed for later data = idata.data; // the bitmap itself // get lumas and build histogram for(i = 0; i < data.length; i += 4) { var luma = Math.round(rgb2luma(data, i)); hgram[luma]++; // add to the luma bar (and why we need an integer) } // find tallest bar so we can use that to scale threshold for(i = 0; i < width; i++) { if (hgram[i] > maxH) maxH = hgram[i]; } // use that for threshold thold *= maxH; // find min value for(i = 0; i < width * 0.5; i++) { if (hgram[i] > thold) { min = i; break; } } if (min < 0) min = 0; // not found, set to default 0 // find max value for(i = width - 1; i > width * 0.5; i--) { if (hgram[i] > thold) { max = i; break; } } if (max < 0) max = 255; // not found, set to default 255 scale = 255 / (max - min) * 2; // x2 compensates (play with value) out.innerHTML = "Min: " + min + " Max: " + max + " Scale: " + scale.toFixed(1) + "x"; // scale all pixels for(i = 0; i < data.length; i += 4) { data[i ] = Math.max(0, data[i] - min) * scale; data[i+1] = Math.max(0, data[i+1] - min) * scale; data[i+2] = Math.max(0, data[i+2] - min) * scale; } ctx.putImageData(idata, 0, 0) } tv.oninput = function() { v.innerHTML = (tv.value * 100).toFixed(0) + "%"; ctx.drawImage(img, 0, 0); process(img, +tv.value) }; function rgb2luma(px, pos) { return px[pos] * 0.299 + px[pos+1] * 0.587 + px[pos+2] * 0.114 } 

 <label>Threshold: <input id=tv type=range min=0 max=1 step= 0.01 value=0.1></label> <span id=v>10%</span><br> <canvas id=c></canvas><br> <div id=out></div> <h3>Original:</h3> <img id=org>