Varnish: how to separately cache pages based on value of a specific cookie

Question

I manage a site that has a single cookie, which we have to use, but will always be one of 9 values (including no value). I'd like to use varnish in front of our application servers, with varnish separately caching a version of each page based on the cookie value.

So if we have page /page1, Varnish should separately manage a copy of what /page1 looks like with cookie values a, b, c, d, etc....

Assume we have plenty of memory on the Varnish server to handle storing all pages with all cookie combinations.

We have tried many VCL settings but can't figure out exactly how to make this work. Varnish needs to send that specific cookie to our application server as well, so our application knows which content to send back.

Thanks in advance!

Answer 1

It's quite simple to achive that in fact, you should add a custom vcl_hash :

sub vcl_hash {
  #...
  /* Hash cookie data */
  # As requests with same URL and host can produce diferent results when issued with  different cookies,
  # we need to store items hashed with the associated cookies. Note that cookies are already sanitized when we reach this point.
  if (req.http.Cookie) {
    /* Include cookie in cache hash */
    hash_data(req.http.Cookie);
  }
  #...
}

With this code, varnish will store a different cookie, for each cookie value... but I'll recommend you to also sanitize your cookies on vcl_recv , this is an excerpt of [1] containing cookie sanitization for Drupal sites:

sub vcl_recv {
  #...
  # Remove all cookies that backend doesn't need to know about.
  # See https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies
  if (req.http.Cookie) {
    /* Warning: Not a pretty solution */
    /* Prefix header containing cookies with ';' */
    set req.http.Cookie = ";" + req.http.Cookie;
    /* Remove any spaces after ';' in header containing cookies */
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    /* Prefix cookies we want to preserve with one space */
    /* 'S{1,2}ESS[a-z0-9]+' is the regular expression matching a Drupal session cookie ({1,2} added for HTTPS support) */
    /* 'NO_CACHE' is usually set after a POST request to make sure issuing user see the results of his post */
    /* Keep in mind we should add here any cookie that should reach the backend such as splahs avoiding cookies */
    set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE|OATMEAL|CHOCOLATECHIP)=", "; \1=");
    /* Remove from the header any single Cookie not prefixed with a space until next ';' separator */
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    /* Remove any '; ' at the start or the end of the header */
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
      /* If there are no remaining cookies, remove the cookie header. */
      unset req.http.Cookie;
    }
  }
  #...
  return(hash);
  #...
}

[1] https://github.com/NITEMAN/varnish-bites/blob/master/varnish3/drupal-base.vcl

Answer 2

NITEMAN provided an excellent answer for the time this question was originally asked. However, the intervening timespan has resulted in an officially-supported VMOD that makes filtering unwanted cookies easier and cleaner. It can be found at: https://github.com/varnish/varnish-modules

From the example in the README (with slight tweaking):

import cookie;

sub vcl_recv {
        cookie.parse(req.http.cookie);
        cookie.filter_except("SESSIONID,PHPSESSID");
        set req.http.cookie = cookie.get_string();
        # Only SESSIONID and PHPSESSID are left in req.http.cookie at this point.
        # ...
        return (hash);
        # ...
}