Remove background color from image in HTML5 Canvas

Question

I'm trying to strip the background out of a photo image loaded into a HTML5 Canvas. Think of something like a green screen effect.

I'm using HTML/JS/jQuery

The background of the photo will be, for example, a green curtain. The color wont be exact across the curtain because of lighting and such like.

What i'm doing right now is grabbing the RGB value of a pixel that the user clicks on within the Canvas. That's then considered to be the background. I add the R+G+B of that pixel to set what is considered as the background.

I'm then going through the canvas, pixel by pixel, checking if the pixel is close to the RGB value set as the background (say within 50 above or below). If it matches, I change the pixel to be transparent within the canvas.

This works well enough as a proof of concept but not well enough to do anything with.

Does anyone have any better ideas on background subtraction?

Cheers!

Answer 1

Have a look at the GrabCut algorithm or GrowCut algorithm ; the former describing 'Foreground Extraction using Iterated Graph Cuts' and the latter, 'Image Segmentation By Cellular Automata'. Both those papers will give you a deeper insight into some of the older algorithms used to remove background image data. If you could somehow implement one of those algorithms in Javascript then I think you're most of the way there.

The OPENcv computer vision library (written in c/c++) has plenty of efficient image manipulation methods to examine. You could try and port one of the OPENcv library's BackgroundSubtractor methods (which I believe is partly based on Chris Stauffer and WEL Grimson's algorithm ) to Javascript and then use that to analyse the background and therefore subtract it, but I think they are based on progressive video frames rather than static images.

The js-aruco project on Google code has ported some of the functionality of the OPENcv library already ( codebase is here ) so you might want to look there first for some inspiration and then, if you're feeling brave, have a look at how you could program the GrabCut or GrowCut algorithms