Varnish Cache and headers

Question

I am trying to cache a certain part of my web app.

I have an endpoint at http://website.dev/pictures/:id that returns PHP generated pictures. Sometimes, the endpoint can come with a width and a height in query string to define picture dimension: http://website.dev/pictures/:id?w=100&h=100 .

So I would like to cache those requests for a very long time.

I tried a very simple VCL as I'm new to it and I don't want to make complicated things:

vcl 4.0;


backend default {
    .host = "127.0.0.1";
    .port = "80";
}

sub vcl_recv {
    if (req.url ~ "^/api/pictures" && req.method ~ "GET") {
        # I heard that it was better to unset the cookies here to allow cache
        unset req.http.cookie;
        return (hash);
    }

    return (pass);
}

sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }
}

But now, I get all MISS on my requests. I tried to modify the responses coming from my backend in order to have appropriate cache-control and expires headers:

Response Headers:
Accept-Ranges:bytes
Age:0
Cache-Control:max-age=10000, private
Connection:keep-alive
Content-Length:96552
Content-Type:image/jpeg
Date:Sun, 30 Jul 2017 16:41:58 GMT
Expires:Fri, 01 Jan 2100 00:00:00 GMT
Server:nginx/1.10.3 (Ubuntu)
Via:1.1 varnish (Varnish/5.1)
X-Cache:MISS
X-RateLimit-Limit:60
X-RateLimit-Remaining:57
X-Varnish:32772

Request Headers
view source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4,es;q=0.2
Cache-Control:max-age=0
Connection:keep-alive
Cookie:_ga=xxxxxxxxxxxxx
Host:website.dev:6081
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36

Where am I wrong here? How could I make sure the requests on such endpoints and only are cached by Varnish for a very long time?

Answer 1

The reason it doesn't cache as expected is because your web application is sending:

Cache-Control:max-age=10000, private

The built-in VCL logic prevents caching in presence of Cache-Control: private .

The best course of action here would be to fix your application: either stick to use of Cache-Control with public type or remove it and keep only Expires header.

If you're not willing to fix the app or there are reasons not to, you'd need to add a bit of VCL. The following will make sure that the "built-in VCL logic" mentioned above will not prevent caching in presence of Cache-Control: private :

sub vcl_backend_response {
    if (beresp.ttl <= 0s ||
      beresp.http.Set-Cookie ||
      beresp.http.Surrogate-control ~ "no-store" ||
      (!beresp.http.Surrogate-Control &&
        beresp.http.Cache-Control ~ "no-cache|no-store") ||
      beresp.http.Vary == "*") {
        /*
        * Mark as "Hit-For-Pass" for the next 2 minutes
        */
        set beresp.ttl = 120s;
        set beresp.uncacheable = true;
    }
    return (deliver);
}

Finally, while your app is sending proper Expires header for caching, it is ignored by Varnish since Cache-Control header has priority over it. (as per HTTP specifications). Take note of it because its value will be used as duration for Varnish to cache the object. (And not value from Expires )