MongoDB Count on millions of row is very slow

Question

Using MongoDB 2.4 with mongoDB .NET driver 3.2

I have a collection with 30 million entries.

var visits = new MongoHelper<CampaignVisitLog>()
         .GetCollection().AsQueryable().Count(t => t.campaignId == campaignId);

campaignId is indexed. Depending on many entries campaignId has it will take from 30 seconds to several minutes to return the count.

Whats the correct way to count this collection?

Answer 1

I have a collection with 30 million entries.

Independently of optimizing the whole query, you can't think that you're going to get blazing-fast responses with millions of items.

If you're performing other queries to get stats, maybe it's the time to schedule this calculations and do them with some asynchronous service (ie a Windows service , Windows scheduled task , Quartz.NET ...), and get their results also asynchronously.

You can either use MongoDB to store your calculation service results or go for a more specific solution: a service bus (ie RabbitMQ , Azure Service Bus , NServiceBus ...).

Answer 2

MongoDB C# driver is sick.

LINQ queries are always translated to aggregation framework pipelines.

var pipeline = [ { "$group" : { "_id" : 1, "__result" : { "$sum" : 1 } } }]
db.test.aggregate(pipeline)

This results in a full collection scan on the server because the LINQ query did not specify any constraints.

Answer 3

ensure you have enough memory to store your index in ram.

https://docs.mongodb.org/manual/tutorial/ensure-indexes-fit-ram/