mongodb performance for large document

Question

I have a document that holds a big data structure in certain fields inside an array, it is slowing down my application due to frequent hits to read such data. am thinking on few solutions to implement but I need advice before i proceed and possibly even a better solution, here are my thoughts/questions:

1. would it help to cache data?
2. should I use memcached or redis as a caching engine and why?
3. would it help to read single fields from this document instead of reading it all every time?
4. should I do something else?!

Answer 1

Caching will help because it would avoid your db to be hit too often

Memcache or redis it's up to you. I prefere redis but if you already have a memcache it's fine.

If you have a cluster of servers, think if you need a centralized cache or not

Caching a full document won't help for getting a single field because you cache the result of a query without knowing what it contains.

Answer 2

your question need more clarification. for example how big is the data that you are speaking of is it couple of megabytes or gigabytes. All these factors change the solution. But if we consider that you have couple of megabytes and you want to prevent to call database every time the best solution is cache. How to choose a cache is also completely depends on what is your situation. If your web application runs on one server you can use the in-memory cache like ASP.Net cache which is very quick and fast for in-memory cache. this cache is stored in your heap so you can put all your object in the cache without serialization.But consider that whenever your application is restarted like most of deployments. your heap will be deleted and all the cache is cleared inside the heap.

if you have more than one server then you can start to think about an out-of-memory cache because two servers are not sharing heap memory and using all in-memory cache are useless because it duplicate the data and invalidating is nightmare. However, this is more reliable cache while it is not in the heap and in term of persistence is more than in-memory cache. But whatever you want to put in this kind of cache should be serializable while you are transferring the object over network connection. So you cannot put all your object in cache. Both Redis and memcached can be used for this purpose. Redis is more complicated with more functionality than Memcached but for your purpose memcached is quite good.

Whatever caching system you choose, approach it in a wide perspective. Design a caching system in your application while over time you need to put more things in cache. so its better to prepare everything for that time from now. another things which is very important in cache is that whenever you set something in cache you have to consider when you are going to invalidate it.

Answer 3

Whether or not caching will help depends on the accession of the document. If the document is being accessed multiple times then caching will not help due to how MongoDB to memory caching actually works.

First, you need to understand your data accession patterns.