Can MongoDB handle TBs of data?

Question

Will MongoDB handle several TB of data? I've read posts saying that Mongo does well with < 1TB of data, for larger sets I should go with HBase. Is that true?

I need to store and later process several TB of text data.

Answer 1

These may be of interest to you:

• Wordnik : data set in the >3TB range
• Craiglist : shard cluster designed to support 10TB of data.

You'll find some additional case studies on 10gen's website, although not all of them provide specific numbers on data set sizes. There are also some older discussions on Stack Overflow about this very question ( see here for a blurb about a user with 12TB of data from March 2010), and you'll likely find more case studies scattered among presentations on Speaker Deck or Slideshare . In short, MongoDB can certainly handle that amount of data (people are using it to that effect today), but you'll want to heed best practices, which is where existing presentations can come in handy.

Answer 2

MongoDB

Tens of thousands of organizations use MongoDB to build high-performance systems at scale. Over a third of the Fortune 100 and many of the most successful and innovative web companies rely on MongoDB. They've grown from single server deployments to clusters with over 1,000 nodes, delivering millions of operations per second on over 100 billion documents and petabytes of data.

Scalability is not just about speed. It's about 3 different metrics, which often work together:

Cluster Scale. Distributing the database across 100+ nodes, often in multiple data centers

Performance Scale. Sustaining 100,000+ database read and writes per second while maintaining strict latency SLAs

Data Scale. Storing 1 billion+ documents in the database There are many examples of MongoDB users who are pushing the limits to scalability. Here are a few, organized around each scaling dimension.

You can find reference about MongoDB: Bringing Online Big Data to Business Intelligence & Analytics at this article