NodeJS GZip Compression - Any Issues?

Question

So I am generally quite new to the MEAN stack and had some questions regarding compression with the NodeJS / express combo.

After digging around I noticed it is not feature that is widely used and it had some issues in the past (zlib) and also has now been separated out in Express 4.0

Having also spotted the documentation for the compression module: https://github.com/expressjs/compression

I have to say, the 'Server-Sent Events' code example raises an eyebrow with the setInterval method. I guess that is mainly for streaming the data rather than giving it one go but still seemed a little strange at first.

So for those who are using this set-up and stack with the above module, can anybody notify me of any gotcha's or issues to be aware of as searching around hasn't given me anything recent.

FYI, I am primarily looking to use it for large amount of JSON transfers and maybe smaller HTML static as well later.

I would also be interested to know what else people are doing for compression if they are not using the above.

Answer 1

I wouldn't put zlib compression on within my application. I would leave it at the web server level where it is highly optimized.

A lot of folks make the mistake of using their Node.js server's HTTP server as their primary serving method. This obviously works, but you can make things more efficient by putting a proper web server, such as Nginx, out in front. Why tie up your application with spoon-feeding slow connections, compression, caching, authentication, etc.

Think of the HTTP server in Node.js as a replacement for CGI/FastCGI. While it can be used as a general purpose HTTP server, you're better off using it as the communication protocol between your application and your web server for most web applications.

(Obviously there are exceptions to this... not all Node.js apps even have anything to do with web serving. Use your best judgement when architecting your app.)

Answer 2

I found node zlib to be quite slow for my intended use.

This is what I tried to use it for:

1. take a multipart form post with attached files
2. pipe it through zLib and crypto (aes-128-ctr) to Amazon S3.

After some timing experiments I found out that the encryption part is quite speedy. It adds only about 5-10% to the total transfer time from client to S3.

ZLib on the other hand adds about 50% to the transfer time. I ended up not compressing the files. Storage on S3 is cheap.