如何最有效地搜索数据？ （RAM或MongoDB中的对象）

Question

In a scenario where your nodeJS server has a static collection of 40mb of text loaded in memory,

Everytime your webclient asks for a search (1000 elements to look through), which method would be optimal?

• Query the mongoDB database
• Execute a .find() on the collection already in RAM that will parse through the 1000 elements looking for multiple different strings in that element

I am looking for the most performant method before I start implementing a complex .find() function.

Answer 1

As @Storm mentioned,

In a scenario where your nodeJS server has a static collection of 40mb of text loaded in memory

So I suppose the data will not be changed after loaded.

Before we choose any technical solution, I suggest you to do a performance load test with common MongoDB settings. Based on below conditions(you didn't mention):

1. The volumn of requests from client(eg 1K/s, 10K/s,...)
2. The target response time(eg 95% of all less than 10ms)
3. What queries you will perform?

Since 40mb 1000 documents is not a large data, MongoDB prefer to put hot data into memory to speedup the queries. If the common MongoDB settings can not satisfy your performance goal, you can consider:

1. Improve the infrastructure software

• Try MongoDB In-Memory engine (Since the data will not be modified)
• Try other memory based DB such as Redis(If you don't need very complex query)

2. Improve the code quality of your usage

• Build efficient indexes in MongoDB
• Any better data model design?