How to fetch all data quickly from mongodb using indexing field without using limit

Question

I have around ~2 million documents in my database, when i made an query in mongo for which looks like:

db.collection.ensure_index("location")
db.collection.find({"location.name": {"$regex": 'norway', "$option": 'i'}}, {"_id:0", 'userid': 1, 'username': 1})

mongo document sample for location searching:

Mongodb document: {"location": {id:3235726, name: Oslo, Norway }}

In indexing i am not specified cache_time part, it uses default. Now, it shows me data after 2 or 4 seconds which is very bad, and the most worse part when it doesn't find any location then it took around 25seconds for searching operation. I mapped this time using python time.time() method. Then i put a limit() on query only for 100 documents, then it works a bit well. But what is the use case for me if i put a limit there, using skip() is very expansive while searching.

On other part if i am searching data using userid like, it doesn't contain regex type query:

db.collection.ensure_index("userid")
db.collection.find({"userid": 1213444}, {"_id:0", 'user_rank': 1, 'username': 1})

It works very efficiently, as id is integer and very fast. Only difference is i am using ' like ' query in location query part with regex, and i studied on mongo docs regex is not bad in terms of performance.

Should i go for increasing physical memory on my server which is currently 1.5G for using indexing efficiently or using Apache Solr search engine which index all my mongo docs through mongo_connector and sync data from mongodb to solr (Indexing data directly to solr is very expansive in terms of memory )?

I tried to put all efforts in my problem statement, if there is anything wrong with my problem statement then let me know, i can try my best to improve it.

Edit: Should i go for elasticsearch with mongo? because with index or without index search result is too slow, like maximum time it shows 260ms which is too high

Answer 1

$regex does not always use index

$regex can only use an index efficiently when the regular expression has an anchor for the beginning (ie ^) of a string and is a case-sensitive match. Additionally, while /^a/, /^a. /, and /^a. $/ match equivalent strings, they have different performance characteristics. All of these expressions use an index if an appropriate index exists; however, /^a. /, and /^a. $/ are slower. /^a/ can stop scanning after matching the prefix.

You have to create query to use index. You can check that index was used with explain command