Server- and client-side caching of a relatively large JSON file

Question

I need to generate and deliver a relatively large JSON file to users. By large, I mean 400 KB. It's the join of two tables, and it will probably get larger over time. I'm planning on using it to fill in the gaps in an HTML table, linking up items via IDs from the database.

To save bandwidth, I only wanted the client to download this JSON file once per day. (The easy options seem to be "every time" and "never"... grr.) In reality, it will update less often server-side, but I don't want it less frequent. 90% of the browsers hitting this are IE7/8, if that makes a difference.

For server-side caching, I'm currently just dropping the JSON file into a text file on the server if it's older than a few hours. I understand I could use memcached instead. Does memcached have a considerable advantage over a file? (Single server, 1k visitors per day, I don't have memcached installed yet.)

Per the PHP manual comments, I tried this:

$expires = 60*60*24;
header("Pragma: public");
header("Cache-Control: maxage=".$expires);
header('Expires: ' . gmdate('D, d M Y H:i:s', time()+$expires) . ' GMT');

But it appears to do nothing. Chrome is informing me that it is "OK" each and every time I load the generating page directly, and no headers regarding cache control show up under the network/headers tab. I'm not explicitly hitting refresh. Doesn't look like I'm caching at all. Are there any other ways to save the client some time and bandwidth? Best idea I have is creative tinkering with the .ajax() cache option.

I haven't actually gotten as far as implementing the Ajax call yet. Baby steps.

(Sorry if I'm a bit wordy. Yell at me and I'll improve the question as much as I can when it is needed. I've done some searching here and on Google and haven't come across anything super useful yet. Maybe I don't have the right key terms.)

Note: I'm running PHP under IIS 7, so it's CGI. I feel like I read that might make a difference.

Answer 1

Sending the correct cache headers is only half the steps needed. You still need to send 304 status saying nothing has changed since the last time the client got the content. This can get tricky. I recommend using .htaccess to do this caching instead. More can be found here

Example

ExpiresActive On
ExpiresDefault "access plus 1 day"

Edit--

If you had to do it all in php then you have to manually check the request headers and send 304. This how you can do it:

$last_modified_time = filemtime($file); 
$etag = md5_file($file); 
header("Last-Modified: ".gmdate("D, d M Y H:i:s", $last_modified_time)." GMT"); 
header("Etag: $etag"); 
if (@strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) == $last_modified_time || 
    trim($_SERVER['HTTP_IF_NONE_MATCH']) == $etag) { 
    header("HTTP/1.1 304 Not Modified"); 
    exit; 
}

Answer 2

Have you thought of using browser local storage? You use javascript to check if the data is stored locally. If it's not, or it's old, then you fetch it from the server and store it locally. Almost all "modern" browsers support local storage, meaning IE8, Firefox 3.5, Safari 4, Chrome and above. Sorry, but IE7 does not support it.

The biggest advantage to memcache is that it is a distributed storage system. So you could have 4 load balanced web servers all accessing the same data store. That it's memory based means better performance, but it doesn't seem like you would be having performance issues.

Answer 3

to save bsndwith, you can also use compresion on the transmission. start the php file with a call to ob_start('ob_gzhandler'); but make sure your content-type is set to text/plain or text/html, because ie7 doesn't think gzipped application/json content exists