如何使用aiohttp向外部服务器发出客户端请求以避免缓存

Question

We are using aiohttp to make multiple requests to various website vendors to grab their latest data.

Some of the content providers serve the data from a cache. Is it possible to request the data from the server directly? We have tried to pass in the headers parameter with no luck.

async def fetch(url):
    global response
    headers = {'Cache-Control': 'no-cache'}
    async with ClientSession() as session:
        async with session.get(url, headers=headers, proxy="OUR-PROXY") as response:
            return await response.read()

The goal is to get the last-modified date header , which is not provided from the cache request.

Answer 1

Try to add some additional variable with dynamic value to URL (eg timestamp). This will prevent caching on the server side even if it ignores Cache-Control.

Example:
from: https://example.com/test
to: https://example.com/test?timestamp=20180724181234