Varnish (Turpentine): Reload page before flushing in Magento

Question

Varnish is really fast in Magento 1.8. The problem is the refresh of a page in the cache.

For example: If something on the index page is changed, I'm using this command to flush:

Mage::getModel('turpentine/varnish_admin')->flushUrl("^/$");

And then this script to reload the page to the cache:

$header[0] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; 
$header[] = "Accept-Language: de,en-US;q=0.7,en;q=0.3"; 
$header[] = "Cache-Control: max-age=0"; 
$header[] = "Connection: keep-alive"; 
$header[] = "Keep-Alive: 300"; 
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; 

$curl = curl_init(); 
curl_setopt($curl, CURLOPT_URL, $url); 
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0'); 
curl_setopt($curl, CURLOPT_HTTPHEADER, $header); 
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate'); 
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$html = curl_exec($curl);
curl_close($curl);

It is working, but the index page with a lot of products is really slow, when it is not cached. So every user see this slow page during the reload.

Is there any solution for this problem? Is it possible to reload the page before flushing it?

Answer 1

You need to implement Grace Mode (aka stale while revalidate).

In your case it will allow Varnish to serve stale copy of your index page after you have already issued the flush request.

Grace mode is a feature to mitigate thread pile-ups, that allows Varnish to continue serving requests when the backend cannot do it.

... or slow