Mysql (fulltext?) search for products

Question

I am building a very simple product catalog that will store products inside mysql table and I want to search products as fast as possible (and as relevant as possible). The products database will be quite large (about 500.000 products) which is why searches using "like" that are not using indexes are very slow.

I have many fields but the only ones I want to search are:

• product_id = bigint
• title = varchar(255)
• description = text

I tried experimenting with fulltext search but there were some problems:

• I couldn't search by product_id since it is big integer and can not be indexed inside fulltext index (sometimes user knows the ID of the product)
• if I search for "Meter XY-123" I get no search results even though the single product's title as well as description contains both words ("meter" and "xy-123")
• I couldn't search for substrings - eg if product's title is "Foobar 123" it should be returned even if user searches for:
  • foo bar 123
  • bar 123
  • foobar 12
  • foo
  • etc.
• results should also be returned ordered by some kind of relevance.. eg if I have two products "foobar 123" and "foobar 456" and user searches for "foobar 4" then both products should be returned (match any word) but second product should be ranked higher (because it contains also number 4) than the first one (that doesn't contain number 4).
• products should also be ranked based on which field the value is found in. In this case product_id field has bigger weight than title which has also higher weight than description. Eg if user searches for "1234" then:
  • first ranked product should be the one that has product_id 1234
  • afterwards there should be ranked products that include "1234" inside the title
  • afterwards products that include this number inside description

What would be the best way to do searches on this table like this? The only way that gives results that are good in my case is splitting the query string and querying multiple queries using "like" operator for every string and somehow calculate the weight but this solution works very slow (even more than 15 seconds for a single query which is too slow).

I don't expect everything to be possible using only single query but I am looking for a solution that would be fast and as relevant as possible. If this means building some kind of custom word index or similar I am also willing to do this, I just need an idea how to manage this?

thank you!

Answer 1

我们将搜索迁移到Sphinx。现在，我们需要微调结果。