Caching Video On-Demand

Question

I would like to cache Video On-Demand from a particular website. When you open the page for the video, it first loads the player, and when you click play it loads the playlist file that contains links to chunklists (media playlists) for different bitrates, these chunklists contain links to video files (media segments).

Just an illustration:

For any of these files I never get a HIT - always a MISS . Because there is Cache-Control: no-cache header field, I tried reply_header_access Cache-Control deny all and reply_header_replace Cache-Control max-age=3600 but this does not help. Then I thought that cookie in the request header might have something to do with it - but no: How does Squid deal with Cookies?

I have noticed that playlist's Content-Length field varies from 201 to 205 - maybe that has something to do with it? But the content inside is always pretty much the same - the only thing that can change the size is that inside there are links to chunklists that have a random number in between ( chunklist_w[random-number]_b800000.m3u8 ), length of a random number is usually 9 or 10 digits. Why I would like to serve playlist file from cache is so browser would always request for the same URL and that URL would be cached so no need to rewrite URLs.

So theoretically the playlist file should be requested only once from the origin (first request to the proxy) and then always from squid cache as well as the chunklists and media files because the URLs would stay the same - so no dynamic content.

Why doesn't squid serve this content from cache? (On the other hand, "static" content [images, js, css] is served from cache)

---

Update #1: I have also tried by expanding the maximum_object_size from 4 MB to 20 MB even though all files are around 2 MB. Still nothing.

---

Additional Info:

With Squid:

random number = 1887187059 (10 digits)

Request Header:

GET [omitted] HTTP/1.1
Host: [omitted]
Proxy-Connection: keep-alive
X-Requested-With: ShockwaveFlash/21.0.0.197
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36
Accept: */*
Referer: [omitted]
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,sl;q=0.4
Cookie: PHPSESSID=jfg6j07l9i6bep37jsacig84e1; nocache=0

Response Header:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Server: WowzaStreamingEngine/4.3.0
Cache-Control: max-age=86400
Date: Sat, 09 Apr 2016 14:45:53 GMT
Content-Type: application/vnd.apple.mpegurl
Content-Length: 205
X-Cache: MISS from squid3
X-Cache-Lookup: MISS from squid3:3128
Via: 1.1 squid3 (squid/3.3.8)
Connection: keep-alive

Without Squid:

random number = 204546888 (9 digits)

Request Header:

GET [omitted] HTTP/1.1
Host: [omitted]
Connection: keep-alive
X-Requested-With: ShockwaveFlash/21.0.0.197
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36
Accept: */*
Referer: [omitted]
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,sl;q=0.4
Cookie: PHPSESSID=jfg6j07l9i6bep37jsacig84e1; nocache=0

Response Header:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Server: WowzaStreamingEngine/4.3.0
Cache-Control: no-cache
Date: Sat, 09 Apr 2016 14:59:56 GMT
Content-Type: application/vnd.apple.mpegurl
Content-Length: 203

Answer 1

The line that made the content cache is:

refresh_pattern \.(ts|m3u8)$ 120 50% 180 override-expire override-lastmod ignore-no-cache

This line tells the cache manager that files with extension .ts or .m3u8 should be considered fresh up to 3 hours and that it should override three HTTP header fields: Expire, Last-Modified and Cache-Control:no-cache

To learn more about refresh_pattern , check Squid Documentation .

Why this content was not cached before is that options override-expire , override-lastmod , ignore-no-cache were missing because in the caching matter time does not really matter - it matters when we want to serve the content to the user.