Mongodb : Pre-compute Raw Collection to multiple (10k+) smaller collections?

Question

I have a raw collection with thousands (100k) of "raw events" only string data.

I need to access data using filter and multiple patterns. All pattern will be known by me (using URL on my website).

Logically, I need to pre-compute all datas into smaller database (ie: collection for mongodb).

This computing will happen only ONE time, the same time the event is added to my large "raw data collection", no real time computing asked by user.

Is it good practice to compute all this events into multiple small collections?

Example:

db.collection.rawdata :

Event 1 : UserA create a party "Group1".
Event 2 : UserA invite UserB to join "Group1".
Event 3 : UserB refuse to join "Group1".
Event 4 : UserA invite UserC to join "Group1".
Event 5 : UserC accept to join "Group1".

I will have:

db.collection.rawdata (max data : 100k documents, only 1 collection).

db.collection.groupeXXX (max data per collection : 2k documents, number collection : 250).

db.collection.UserXXXXX (max data per collection : 1-500 documents, number collection : 10k)

db.collection.InvitationAccepted (max data per collection : 5k documents, only 1 collection)

db.collection.InvitationRefused (max data per collection : 5k documents, only 1 collection)

So basically a collection for every user, a collection for every group, and collections for every event type. When a user ask for data on my website, I just have to display all documents from a specific collection.

Answer 1

When a user ask for data on my website, I just have to display all documents from a specific collection.

Often ideas like this start out well and as the application becomes more complex and gets additional requirements to connect various data points, make it awkward to implement those requirements.

Start with a schema that makes sense for the data you are storing and operations you are performing. IF you have specific performance needs, measure your performance and optimize the aspects that need to be optimized.