Web worker out of memory in HTML5 canvas processing

Question

In Main thread : I get the source image array by getImageData method. It is a uint8ClampedArray to record my image data.

Here is my code in web worker : (this code will give me a high resolution image but the result of operation is too huge)

 self.addEventListener('message', function(e){
    var scale = e.data['scale'], cvWidth = e.data['width'], cvHeight = e.data['height'], sBufferData = e.data['sBuffer'].data;

    var sq = scale*scale,   //pixel of source within target
        sw = cvWidth, sh = cvHeight,
        sx = 0, sy = 0, sIdx = 0,
        tw = Math.floor(sw * scale), th = Math.floor(sh * scale),
        tx = 0, ty = 0, tIdx = 0,
        TX = 0, TY = 0, yIdx = 0,
        w = 0, nw = 0, wx = 0, nwx = 0, wy = 0, nwy = 0,
        sR = 0, sG = 0, sB = 0,
        crossX = false, crossY = false,
        tBuffer = new Float32Array(3 * sw * sh);

    for( sy=0; sy<sh; sy++ ) {
        ty = sy * scale; TY = 0 | ty; yIdx = 3 * TY * tw; crossY = (TY != (0 | ty + scale));
        if(crossY) { wy = (TY + 1 - ty); nwy = (ty + scale - TY - 1); }
        for( sx=0; sx<sw; sx++, sIdx+=4 ) {
            tx = sx * scale; TX = 0 | tx; tIdx = yIdx + TX * 3; crossX = (TX != (0 | tx + scale));
            if(crossX) { wx = (TX + 1 - tx); nwx = (tx + scale - TX - 1); }
            sR = sBufferData[sIdx+0]; sG = sBufferData[sIdx+1]; sB = sBufferData[sIdx+2];
            if(!crossX && !crossY) {
                tBuffer[tIdx+0] += sR * sq; tBuffer[tIdx+1] += sG * sq; tBuffer[tIdx+2] += sB * sq;
            } else if(crossX && !crossY) {
                w = wx * scale;
                tBuffer[tIdx+0] += sR * w; tBuffer[tIdx+1] += sG * w; tBuffer[tIdx+2] += sB * w;
                nw = nwx * scale
                tBuffer[tIdx+3] += sR * nw; tBuffer[tIdx+4] += sG * nw; tBuffer[tIdx+5] += sB * nw;
            } else if(crossY && !crossX) {
                w = wy * scale;
                tBuffer[tIdx+0] += sR * w; tBuffer[tIdx+1] += sG * w; tBuffer[tIdx+2] += sB * w;
                nw = nwy * scale
                tBuffer[tIdx+3 * tw+0] += sR * nw; tBuffer[tIdx+3 * tw+1] += sG * nw; tBuffer[tIdx+3 * tw+2] += sB * nw;
            } else {
                w = wx * wy;
                tBuffer[tIdx+0] += sR * w; tBuffer[tIdx+1] += sG * w; tBuffer[tIdx+2] += sB * w;
                nw = nwx * wy;  //for TX + 1; TY px
                tBuffer[tIdx+3] += sR * nw; tBuffer[tIdx+4] += sG * nw; tBuffer[tIdx+5] += sB * nw;
                nw = nwy * wx;
                tBuffer[tIdx+3 * tw+0] += sR * nw; tBuffer[tIdx+3 * tw+1] += sG * nw; tBuffer[tIdx+3 * tw+2] += sB * nw;
                nw = nwx * nwy;
                tBuffer[tIdx+3 * tw+3] += sR * nw; tBuffer[tIdx+3 * tw+4] += sG * nw; tBuffer[tIdx+3 * tw+5] += sB * nw;
            }
        }
    }

    var worker2Object = {'tBuffer': tBuffer}; //I guess tBuffer is too Large
        tBuffer = undefined; e = undefined;

    self.postMessage(worker2Object);   //Return to Main Thread

}, false);

It will be out of memory in web worker thread when I using the IE or FireFox (Chrome is fine).

But It good to work on 3 browsers when the image size less then 1MB.

Therefore I suspect the problem is the result of the operation (tBuffer) is too large . Can anyone to help me to solve this problem? thanks for your help.

I need to move the big-O(n^2) algo to web worker then it will not block my main thread to execute the other scripts.

Now, I need to scale 5 spec image in 5 web worker thread. procedure code below

for( var i in 5 spec image size array ) { //LOOP 5 time and new 5 web workers
   //get imageData of source image
   sourceBuffer = canvas.getContext("2d").getImageData(0, 0, w, h); 

   //new Worker_1 to process spec_1 image and so on
   Worker_1 = new Worker('myWebWorker.js');
   Worker_1.postMessage(sourceBuffer);
   Worker_1.onmessage = function(e){ get e.data }
}

Answer 1

Well, here is a thought - when you use Float32Array remember that each color component is converted from 8-bit (1 byte) size to 32-bit size (4 bytes).

So if your image is, lets say 1000x1000 for simplicity the memory consumption would be:

Normal canvas:
1000 x 1000 x 4 = 4,000,000 bytes or 3.8 mb

In case of 32-bits float array the size would be:

Float32Array:
1000 x 1000 x 16 = 16,000,000 bytes or 15.3 mb

Now multiply that with 5 (assuming each works on the image or image with same size) and your initial 3.8 mb canvas (raw data) would be 76.5 mb using float32.

If you image then is more in the lane of 2000 x 2000 then you are at 305 mb + 15 mb for canvas, or 4000 x 4000 then 1.2 gb (!) + 61 mb for canvas and so on.

Additionally: the raw size is just for the buffer you have access to. The browser may or may not have a back buffer of that data which means in the latter case you will be well over into gigabyte sizes which could easily explain the out of memory error message.

If you need high-precision color handling (which I assume since you're using floats) an alternative could be to:

• use 16-bits unsigned buffers instead. This will reduce the size to 50% of what you have now
• or if possible use 8-bit buffer and process the data offline in floats before putting the result back into the buffer
• or use segmented processing using float buffers but only process a few lines of the image each time.

Tip: you will get a huge performance gain if you use transferable objects with the workers.