How to deflate a TypedArray uzing zlib in node.js?

Question

For some Int32Array a , I have tried this:

var d = zlib.deflateSync(new Buffer(a));

But when I do

var b = new Int32Array(zlib.inflateSync(d));

I get b which is different than a .

I am aware of idea of turning the array into a string before deflating. This is just not memory efficient.

Is there a way to do this efficiently?

Answer 1

The main problem here is not in deflating and inflating, as those two actions just transfer data from one Buffer element to another Buffer element, and vice versa, as expected. Problem here is how to transfer typed array (eg Int32Array ) to Buffer , and from Buffer back to typed array. To do that we have to deal with Buffer , ArrayBuffer , and typed arrays here.

Let's have Int32Array a to start with:

var a = new Int32Array([3,-2,258]);

If you create a Buffer from this typed array directly with var b = new Buffer(b); , you will get a buffer which contain 3 bytes instead of 12 (each element in Int32Array has 4 bytes), which means you will lose some information. The right way to do this is to create Buffer from ArrayBuffer . You can do it like so:

var b = new Buffer(a.buffer);

That way you get a Buffer which contains 12 bytes. Now it is easy to deflate and inflate this buffer b :

var c = zlib.deflateSync(b);
var d = zlib.inflateSync(c);

Now buffers d and b contain the same elements. Nothing special about this.

Now if you create Int32Array directly from Buffer d , you will get 4 times more elements that you should because each byte in the Buffer d will be mapped to one element in Int32Array . To overcome this problem, we will use Uint8Array :

var e = new Uint8Array(d);

We used Uint8Array just to map each byte of the buffer d to one byte of an typed array of which we now have access to its ArrayBuffer . The last step is to create the final Int32Array from ArrayBuffer of typed array e :

var f = new Int32Array(e.buffer);

Finally we have f that looks the same as a .

Note that some steps above depend on Endianness . If you do all this on the same machine you should be fine with the steps above.