NGINX 速率限制在使用 Cloudflare 时不起作用。 我可以用一个简单的 `ab` 命令关闭我的网站

Question

I implemented a pretty simple but super effective rate limiting based on this blog post: https://www.nginx.com/blog/rate-limiting-nginx/

Basically:

limit_req_zone $binary_remote_addr zone=ip:10m rate=10r/s;

limit_req zone=ip burst=20 nodelay;

It works great. However, recently I tried Cloudflare, and this doesn't protect me anymore. I can bring down the site myself with a simple command of:

ab -k -c 1000 -n 10000 site.com/

What's happening?

Answer 1

ab -k -c 1000 -n 10000 site.com/ is running 1000 requests in parallel, until a total of 10 000 requests total have been done.

That's too brutal. It's likely that neither the client nor the server are tuned to handle thousands of connections over a few seconds.

Adjust the nginx configuration and do a gentle test ab -k -c 5 -n 500 site.com/

limit_req_zone $http_cf_connecting_ip zone=ip:10m rate=3r/s;
limit_req zone=ip;

limit_conn_status 429;
limit_req_status 429;

429 Too Many Requests

This configures nginx to return the standard status code 429 Too Many Requests when requests are rejected due to rate limiting.

nginx returns a 503 error by default (a bad default) meaning the application is failing, but it is not failing it is rate limited. It's important to configure status code appropriately to distinguish between server errors and rate limiting.

Cloudflare and client IP

When behind cloudflare, nginx will not see the IP of the client but the IP of the cloudflare server. One might think that it breaks rate limiting by IP but it does not, well, just a bit.

When testing locally with ab , your test computer is only resolving a handful of cloudflare servers, and ab probably only uses the first IP. So no there aren't numerous clients IP, the rate limiting should work just fine.

When in production , there will be different clients accessing through different cloudflare servers. Still, there aren't that many cloudflare servers and clients in a geographic area will most likely resolve to the same cloudflare servers. So there will be a bunch of different IPs somewhat defeating the rate limiting, but probably not that many.

> nslookup mycloudflaresite.com

Name:    mycloudflaresite.com
Addresses:  104.28.14.125
            104.28.15.125
            2606:4700:3037::681c:e7d
            2606:4700:3036::681c:f7d

Cloudflare puts the original client IP in the CF-Connecting-IP header. It can also be in the X-Forwarded-For header or X-Real-Ip or True-Client-IP depending on settings and requests. See https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-

Hence the above configuration does rate limiting by client IP using the CF-Connecting-IP header. The nginx variable $binary_remote_addr would be the cloudflare server IP.

Do not use X-Forwarded-For to rate limit

The X-Forwarded-For header can be controlled by the client. It shouldn't be used for rate limiting because it is trivial to circumvent.

Example with a client having the IP 100.11.22.33 :

• On a request without a X-Forwarded-For header => Cloudflare sets X-Forwarded-For: 100.11.22.33 and CF-Connecting-IP: 100.11.22.33 on the request.
• On a request with a X-Forwarded-For: dummyvalue header already set => CloudFlare sets X-Forwarded-For: dummyvalue,100.11.22.33 and CF-Connecting-IP: 100.11.22.33 on the request.

As you can see, it's trivial for the client to put a random value per request and totally circumvent any rate limiting based on the X-Forwaded-For header.

Answer 2

I would suggest trying the following:

(not sure if your backend is PHP but you can readjust)

# Conection Limit, taking the best from:
# http://serverfault.com/questions/177461/how-to-rate-limit-in-nginx-but-including-excluding-certain-ip-addresses
# http://gadelkareem.com/2015/01/18/limiting-connections-requests-wordpress-nginx/

geo $whitelist {
   default 0;
   # CIDR in the list below are not limited ( 1 )
   127.0.0.1/32 1;

}

map $whitelist $limit {
    0     $binary_remote_addr;
    1     "";
}

# The directives below limit concurrent connections from a 
# non-whitelisted IP address to five

limit_conn_zone      $limit    zone=conlimit:30m;
limit_conn_zone      $limit    zone=conlimit_php:30m;

limit_conn           conlimit 40;
limit_conn_log_level warn;   # logging level when threshold exceeded
limit_conn_status    503;    # the error code to return

# Limit Req Non - PHP 
limit_req_zone       $limit   zone=reqlimit:30m  rate=5r/s; 
limit_req            zone=reqlimit burst=10;

# Limit Req PHP
limit_req_zone       $limit   zone=reqlimit_php:30m  rate=1r/s; 

limit_req_log_level  warn;
limit_req_status     503;

# Location VirtualHost

limit_req            zone=reqlimit_php burst=4;
limit_conn           conlimit_php 10;

And for CF-Connecting-IP , you can whitelist the Cloudflare IPs via:

#!/bin/bash
cloudflare_real_ip_conf='/usr/local/nginx/conf/cloudflare_ip.conf'

echo "#Cloudflare" > ${cloudflare_real_ip_conf};
for i in `curl -sk https://www.cloudflare.com/ips-v4`; do
        echo "set_real_ip_from $i;" >> ${cloudflare_real_ip_conf};
done
for i in `curl -sk https://www.cloudflare.com/ips-v6`; do
        echo "set_real_ip_from $i;" >> ${cloudflare_real_ip_conf};
done

echo "" >> ${cloudflare_real_ip_conf};
echo "# use any of the following two" >> ${cloudflare_real_ip_conf};
echo "real_ip_header CF-Connecting-IP;" >> ${cloudflare_real_ip_conf};
echo "#real_ip_header X-Forwarded-For;" >> ${cloudflare_real_ip_conf};

which will produce the CF whitelist (currently):

#Cloudflare
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2a06:98c0::/29;
set_real_ip_from 2c0f:f248::/32;

# use any of the following two
real_ip_header CF-Connecting-IP;
#real_ip_header X-Forwarded-For;

Answer 3

You can add below to you nginx config

real_ip_header CF-Connecting-IP;

See the below article for more details

https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-