Load Balancing Long polling requests with Node.js and Nginx

Question

What is the best load balancing strategy and software I can use to support about 50 Million users each with Http Long Polling/Websockets requests (this is important). I am trying out the Node.js + Nginx option right now, but I have also heard of Erlang. Is there a solution which has been proven to handle such load?

Update - I had a typo in there. The concurrency is expected to be about 50 Million and not 500 Million.

Answer 1

First, 500 million is 1.6 times the number of people in the US. This number is very high, perhaps even excessively high. If you really need to scale to that number, you need a strategy for getting more than a single machine on the job. No machine by itself can really handle that. The TCP socket buffers alone would easily break the Terabyte memory range, especially since you want to employ long polling. I have a hunch that your request is higher than what Google is experiencing for standing connections.

If you want to do it, no single technology will do it. You need to employ several different subsystems and you need to think about how data is stored, how data is moved around and so on. Given a sizeable cluster, I think Node.js or Erlang are really good choices as pieces of the puzzle. But this won't address the question of storing data for 500 million users.