Serving High Traffic Video Over Nginx

Question

I am trying to serve a single High-Res video to 30+ clients exactly concurrently . This is leading to a bottle-neck causing some clients to time-out, and some to experience considerable lag.

Right now – it is clear that each individual request is being processed separately so the server is sending many gigs/sec. Someone mentioned it may be possible to configure Nginx to recognize that the requests are all for the same asset, serve it once, and then let the router (we are running this server locally) send the file to all the devices.

Is this feasible and/or is there any other way to increase my throughput?

Here's my conig per request. It's very simple so far...

    upstream unicorn {
      server unix:/tmp/unicorn.todo.sock fail_timeout=0;
    }

    server {
        listen 80 default;
        root    /usr/local/var/rails/todo-after/public;

        try_files $uri/index.html $uri $uri/video @unicorn;
        location @unicorn {
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_pass http://unicorn;
        }

        error_page 500 502 503 504 /500.html;
        client_max_body_size 4G;
        keepalive_timeout 10;
    }

Answer 1

Honestly self-hosting something like this isn't ideal, I recommend hosting the video on a CDN. They can better handle things like heavy bandwidth and co-location for better latency.

Check out Amazon S3 or Rackspace CDN

EDIT

It appears as though Nginx has some convenience functionality for streaming video. You can limit the bandwidth on a single download to a multiple of the bit-rate of the video.

location /video/ {
  mp4;
  mp4_limit_rate        1.2; #1.2 times the bitrate of the video.
  mp4_limit_rate_after  15s; #After downloading 15s of video.
}

Answer 2

S3 as a origin data storage is good enough. But to serve out of S3 directly cost you a lot.

Two options you have: CDN (CloudFront with S3 as origin) or Nginx S3 proxy.

First one is easy to setup, very efficient, geographically distributed but doesn't support specific features like URL masking, HTTP2/0 with multiplexing (aka SPDY), custom caching and custom authentication.

The second option is the opposite and provide you full freedom. Nginx S3 proxy is very strong, cheap and expenses friendly, just imagine someone tries to fetch a big video file over CDN thousand times, the bill increases proportionally. For some projects is not a problem.

So you choose what suits you better.

PS I recommend to serve big files on a separate machine with bigger upper limits for open files + utilise more Nginx workers.