过滤搜索查询的最佳方法是什么 - PHP MySQL

Question

Im building a site where users can search the posts. Each post is stored inside a database. When a user makes a search eg iPad Mini FOR SALE the query will look like:

SELECT * FROM testtable WHERE title REGEXP 'iPad|Mini|FOR|SALE'

The query will then result to these 3 items:

• Selling iPad Mini
• Selling iPad
• Looking for authentic Gold Watches

The search was successful with the first two items but the third item really just doesn't belong in the group. I want to filter it out and just show relevant items to the search. I'm thinking of taking out the common words like for, is, are, etc. but maybe you guys have any suggestions?

Side note: Do you guys recommend "REGEXP"? I just saw it, used it, and didn't dive into it yet. (No need to answer this just the search filter question, but if you have some good info that would be great.)

Answer 1

You should also take a look at FULLTEXT search. In order to make FULLTEXT search work you need MyIsam MySQL table engine type there are innoDB support too but i don't know much about it.

Answer 2

Already readed this topic? http://www.regular-expressions.info/mysql.html Or this is also a good tutorial / explanation: http://www.tech-recipes.com/rx/484/use-regular-expressions-in-mysql-select-statements/

Answer 3

Yes, remove common words

These are called stop words . These are words that are generally irrelevant.

Consider relevance

A post titled 'ipad mini for sale' is very relevant for a user searching [ipad mini for sale]. A post titled 'ipad for sale' is less relevant. A post titled 'cheese factory for sale' is less relevant still.

Consider deriving an algorithm for calculating what you deem relevant with respect to the posts on your site and the terms searched for by users.

The algorithm may be as simple as looking at the terms searched for and the occurrence of the terms in the post title. Are all terms searched for present in the title? Probably very relevant. Are 10% of the terms searched for present in the title? Probably very irrelevant.

Consider how you want to calculate a relevance score. Set a threshold below which results are deemed too irrelevant to be present in the results. From experience, I'd suggest setting the threshold quite high and aim for highly relevant results only, perhaps listing less relevant results only if no highly relevant results can be found

Use stemming

As an aside, use stemming in your search. A stemming algorithm will reduce a word down to a common stem. You will search for the stem only not the full search term. Read up on stemming . Find an implementation of the porter stemming algorithm for the language you are using; it's a long-standing algorithm and from experience it's fast and pretty much good enough for most applications.

REGEXP?

If you remove stop words and use a stem-based approach, this will be a less relevant concern. In any case, it's a matter of implementation and is likely too subjective a matter to get you a meaningful answer. Try it, examine performance. Try another approach, examine performance. Use what works best for you.