Varnish cache vs backend security?

Question

I am trying to make Varnish work between existing Silex (PHP Symfony) backend which already produces a response with max-age in the header and Drupal frontend. The problems are that:

• Silex adds Cookie to each response
• Drupal adds Authorization string into each request header
• and I've found out that request with Cookies and Authorization headers, in general, miss the cache.

I was trying first to disable this "default" Varnish behaviour by modifying default.vcl file but unfortunately this didn't work until I removed Cookie and Authorization strings from the request.

backend default {
    .host = "172.118.0.1";
    .port = "88";
}

sub vcl_recv {
  if (!(req.url ~ "^/admin/")) {
      unset req.http.Cookie;
  }

  if (req.http.Authorization || req.http.Cookie) {
   /* Not cacheable by default */
   return (pass);
  }
}

My setup:

• 172.118.0.1 backend (Silex)
• 172.18.0.1 front (Drupal)
• 172.19.0.1 Varnish

So here is the request example from Drupal which does not hit the cache and is passed to the backend each time until I remove Authorization line from the header:

*   << Request  >> 17
-   Begin          req 16 rxreq
-   Timestamp      Start: 1518013537.955268 0.000000 0.000000
-   Timestamp      Req: 1518013537.955268 0.000000 0.000000
-   ReqStart       172.19.0.1 49216
-   ReqMethod      GET
-   ReqURL         /silex-api/
-   ReqProtocol    HTTP/1.1
-   ReqHeader      User-Agent: GuzzleHttp/6.2.1 curl/7.38.0 PHP/7.0.26
-   ReqHeader      Host: 172.19.0.1
-   ReqHeader      Content-Type: application/json
-   ReqHeader      Accept: application/hal+json, application/json
-   ReqHeader      x-drupal-run-id: AB1234567890
-   ReqHeader      Authorization: Basic ###############
-   ReqHeader      x-client-ip: 172.18.0.1
-   ReqHeader      X-Forwarded-For: 172.19.0.1
-   VCL_call       RECV
-   VCL_return     pass
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PASS
-   VCL_return     fetch
-   Link           bereq 18 pass
-   Timestamp      Fetch: 1518013538.496350 0.541082 0.541082
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Date: Wed, 07 Feb 2018 14:25:37 GMT
-   RespHeader     Server: Apache/2.4.10 (Debian)
-   RespHeader     X-Powered-By: PHP/5.6.32
-   RespHeader     Cache-Control: max-age=600, public, s-maxage=600
-   RespHeader     x-content-digest: en93a2e062ff1dfe53e166cd7916ac9e44f3ba3d61100ad01a86228dda44b5b125
-   RespHeader     Content-Length: 1596
-   RespHeader     Age: 1
-   RespHeader     X-Symfony-Cache: GET /silex-api/: fresh
-   RespHeader     Content-Type: application/hal+json
-   RespHeader     X-Varnish: 17
-   RespHeader     Age: 1
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   VCL_return     deliver
-   Timestamp      Process: 1518013538.496405 0.541137 0.000055
-   RespHeader     Accept-Ranges: bytes
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1518013538.496544 0.541276 0.000139
-   ReqAcct        281 0 281 447 1596 2043
-   End

So my questions are:

• if I understood the concept of hucking vcl right, and those few lines suppose to force varnish to hit the cache even if there are Cookie or Authorization headers?
• How to implement some security and limit access to backend otherwise if both headers are not cachable?
  • other than IP restriction at the backend?

Thanks in advance for any suggestions.

Answer 1

You are right - Varnish does not cache any response where the request contains cookies or an Authorization header or if the response sets cookies. This is defined in the builtin.vcl: https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishd/builtin.vcl

vcl_receive:

if (req.http.Authorization || req.http.Cookie) {
    /* Not cacheable by default */
    return (pass);
}

vcl_backend_response:

} else if (beresp.ttl <= 0s ||
  beresp.http.Set-Cookie ||
  ...) {
    # Mark as "Hit-For-Miss" for the next 2 minutes
    set beresp.ttl = 120s;
    set beresp.uncacheable = true;
}

Therefore if you want Varnish to cache a request you need to unset the cookies & Authorization header.

Why does Varnish work this way - because the response to a request might differ depending on the cookies and it very likely will differ depending on the Authorization header and is therefore not cacheable - which is exactly what Varnish assumes.

Thus in my opinion you never want to cache a response to authorized request. End of story for me.

If you do have reasons to think differently you have to modify vcl_recv to not return "pass" when an Authorzation header is set. You also need to modify the way Varnish creates the hash under which a response is cached. See: https://varnish-cache.org/docs/5.2/users-guide/vcl-hashing.html

I would not walk down this way.