在SOLR或Elasticsearch中排除记录的好方法

Question

For a matchmaking portal, we have one requirement where in, if a customer viewed complete profile details of a bride or groom then we have to exclude that profile from further search results. Currently, along with other detail we are storing the viewed profile ids in a field (Comma Separated) against that bride or groom's details.

Eg., if A viewed B, then in B's record under the field saw_me we will add A (comma separated).

while searching let say the currently searching members id is 123456 then we will fire a query like

Select * from profiledetails where (OTHER CON) AND 123456 not in saw_me;

The problem here is the saw_me field value is growing like anything, is there any better way to handle this requirement? Please guide.

Answer 1

If this is using Solr:

1. first, DON'T add the 'AND NOT ...' clauses along with the main query in q param, add them to fq. This have many benefits (the fq will be cached)
2. Until you get to a list of values that is maybe 1000s this approach is simple and should work fine
3. After you reach a point where the list is huge, maybe it time to move to a post filter with a high cost ( so it is looked up last). This would look up docs to remove in an external source (redis, db...).

Answer 2

In my opinion no matter how much the saw_me field grows, it will not make much difference in search time.Because tokens are indexed inversely and doc_values are created at index time in column major fashion for efficient read and has support for caching from OS.
ES handles these things for you efficiently.