具有 Nginx、Varnish 和 Django 的多个域都返回相同的缓存页面

Question

I am trying to configure a second domain for an existing project which was previously just using one. But Varnish always returns the cached page from the first domain. So when I visit the second domain I see the content of the first domain. My configurations is as follows:

Note:

• 99% of the configuration was already there
• I changed the domain names and removed some SLL configurations for this post to be more clear.
• Both domains use the same html page but slightly different content.
• I am noob regarding nginx and varnish.

NGINX

server_tokens off;                                                                                             
resolver 127.0.0.53 ipv6=off;                                                                                  
upstream django_app_server {                                                                                   
    server unix:/home/test/run/gunicorn.sock fail_timeout=0;                                                 
}                                                                                                              

#http redirect too https.                                                                                      
server {                                                                                                       
    listen 80 default_server;                                                                                  
    server_name _;                                                                                             
    return 301 https://$host$request_uri;                                                                      
}                                                                                                              

server {                                                                                                       
    server_name existingdomain.com newdomain.com;                                                                                                                                                                 
    listen 443 ssl default deferred;                                                              

    # match with actual application server                                                                     
    client_max_body_size 10M;                                                                                  
    keepalive_timeout 60s;                                                                                     

    # proxy the request through varnish before sending it to gunicorn.        
    location / {
        proxy_pass http://127.0.0.1:6081;
    }
}
server {

    listen 8000;
    server_name existingdomain.com newdomain.com;
    root /home/test/www;

    location / {
        proxy_pass_header Server;
        proxy_redirect off;
        proxy_connect_timeout 60;
        proxy_read_timeout 60;
        proxy_set_header Host existingdomain.com; #changed to $host but results in 127.0.0.1 instead of domains
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_pass http://django_app_server;
    }

    client_max_body_size 10M;
    keepalive_timeout 60s;
}

VARNISH

vcl 4.0;

import std;
import directors;

acl purgers {
    "localhost";
}

backend default {
    .host = "127.0.0.1";
    .port = "8000";
}

sub vcl_recv {

    # uncruft
    unset req.http.User-Agent;
    unset req.http.Accept;
    unset req.http.Accept-Language;

    # Normalize the query arguments
    set req.url = std.querysort(req.url);

    # Fix client IP forwarding
    # Note: this expects Varnish to be behind upstream that sets this securely
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    # allow purge
    if (req.method == "PURGE") {
        if (client.ip ~ purgers) {
            return(purge);
        } else {
            return(synth(403, "Access denied."));
        }
    }
    elif (req.method == "BAN") {
        if (client.ip ~ purgers) {
            # assumes the ``X-Ban`` header is a regex
            ban("obj.http.x-url ~ " + req.http.x-ban);
            return(synth(200, "Ban added"));
        } else {
            return(synth(403, "Access denied."));
        }
    }
    # only head/get
    if (req.method != "GET" && req.method != "HEAD") {
        return(pass);
    }

    # kill cookies for everything else
    unset req.http.Cookie;
    return(hash);
}

sub vcl_backend_response {
    # keep for lurker bans
    set beresp.http.x-url = bereq.url;

    # do the gzip dance with nginx
    if (beresp.http.content-type ~ "^text/" || beresp.http.content-type ~ "^application/json") {
        set beresp.do_gzip = true;
    }

    if (beresp.ttl <= 0s || beresp.http.Cache-Control ~ "no-cache|no-store|private") {
        # mark as "Hit-For-Pass"
    if (beresp.ttl <= 0s || beresp.http.Cache-Control ~ "no-cache|no-store|private") {
        # mark as "Hit-For-Pass"
        set beresp.ttl = 1m;
        set beresp.uncacheable = true;
        return (deliver);
    }
    # stop server error hammering
    if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504) {
        set beresp.ttl = 5s;
        unset beresp.http.Set-Cookie;
        return (deliver);
    }
    # stop 404 hammering
    if (beresp.status == 404) {
        set beresp.ttl = 10s;
        unset beresp.http.Set-Cookie;
        return (deliver);
    }
    # don't cache 40x responses
    if (beresp.status == 400 || beresp.status == 401 || beresp.status == 402 || beresp.status == 403) {
        set beresp.ttl = 5m;
        set beresp.uncacheable = true;
        unset beresp.http.Set-Cookie;
        return (deliver);
    }

    unset beresp.http.Set-Cookie;
    set beresp.grace = 2m;
    set beresp.ttl = 5m;

    return (deliver);
}

sub vcl_deliver {
    # for internal use only
    unset resp.http.x-url;

    # debug info
    if (obj.hits > 0) {
        set resp.http.X-Cache = "hit";
    } else {
        set resp.http.X-Cache = "miss";
    }
    # set resp.http.X-Cache-Hits = obj.hits;

    # cleanup headers
  # cleanup headers
    unset resp.http.Server;
    unset resp.http.X-Varnish;
    unset resp.http.Via;

    return (deliver);
}

sub vcl_purge {
    # only handle actual PURGE HTTP methods, everything else is discarded
    if (req.method != "PURGE") {
        # restart request
        set req.http.X-Purge = "Yes";
        return(restart);
    }
}

I tried:

• Changed NGINX: proxy_set_header Host existingdomain.com; to proxy_set_header Host $host;
• Changed NGINX: 2 server configurations listing to port 8000 for both domains
• Changed VARNISH: 2 backend configurations for both domains

What do I want:

• Eventually I want 2 different domains and several subdomains who all need its own varnish cache.

Answer 1

The following config line:

proxy_set_header Host existingdomain.com;

Sends the same Host: header for both domains.

It should be:

proxy_set_header Host $host;

The other answer mentions $hostname variable which is incorrect, because it stands for machine name. Whereas you want $host as this is equal to value of Host: header from the client.

Answer 2

# proxy the request through varnish before sending it to gunicorn.        
location / {
    proxy_pass http://127.0.0.1:6081;
}

You are not passing any details to varnish. So Varnish can't figure out which domain to work with. Maybe something like

  proxy_set_header        Host $host;
  proxy_set_header        X-Real-IP $remote_addr;
  proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header        X-Forwarded-Proto $scheme;
  proxy_pass              http://127.0.0.1:6081;

Plus like others have stated you need to update what the host you are sending to gunicorn

Answer 3

The following Nginx setting causes Varnish to only serve pages from existingdomain.com , even if other hosts are requested:

proxy_set_header Host existingdomain.com;

Varnish uses both the URL and the hostname to identify objects in cache. When you hardcode the hostname to existingdomain.com in Nginx, you'll always end up with the same content.

Please change this setting to the following value:

proxy_set header Host $hostname;