Where can i cache pandas dataframe in tornado requesthandler

Question

I want to cache a pandas dataframe into tornado requesthandler. So i don't want to repeat the pd.read_csv() for every hit to that particular url.

Answer 1

Since a new RequestHandler is instantiated on each invocation, any data that is supposed to persist between calls needs to be stored somewhere, and CSV is as good a form as any. Alternatively, you can use any of a number of Python caching methods .

Answer 2

Depends on how and where you want to be able to access this cache in the future, and how you want to handle invalidation. If the CSV files don't change then this could be as simple as @functools.lru_cache or a global dict. If you need one cache shared across multiple processes then you could use something like memcached or redis, but then you'll still have some parsing overhead depending on what format you use. In any case, there's not really anything Tornado-specific about this.