MySQL Full text search extremely slow on a AWS RDS large instance

Question

I have a table having 14 million rows and i am trying to perform a full text search on this table. The query for this is performing really slow, it is taking around 9 seconds for a simple binary AND query. The same stuff executes instantly on my private cluster. Size of this table is around 3.1 GB and it contains 14 million rows. Can someone explain this behavior of RDS instance?

SELECT count(*) 
FROM table_name WHERE id=97
AND match(body) against ('+data +big' IN BOOLEAN MODE) 

Answer 1

A high IO rate often indicates insufficient memory, or buffers too small. A 3GB table, including indexes, should fit entirely in memory of a (much-less-than) 500$-per-month dedicated server.

MySQL has many different buffers, and as many parameters to fiddle with. The following buffers are the most important, compare their sizes in the two environments:

If InnoDB: innodb_buffer_pool_size

If MyISAM: key_buffer_size and read_buffer_size

Answer 2

have you added FULLTEXT index on body column if not then try this one surely it will make a big difference

ALTER TABLE `table_name` ADD FULLTEXT INDEX `bodytext` (`body`); 

Hope it helps

Answer 3

Try this

SELECT count(1) 
FROM table_name WHERE id=97
AND match(body) against ('+data +big' IN BOOLEAN MODE) 

This should speed it up a little since you dont have to count all columns just the rows.

Can you post the explain itself?

Answer 4

Since DB version, table, indexes and execution plans are the same, you need to compare machine/cluster configurations. Main points of comparison CPU power available, cores used in single transaction, storage read speed, memory size and read speed/frequency. I can see Amazon provides a variety of configurations, so maybe you private cluster is much more powerful, than Amazon RDS instance config.

To add to above, you can level the load between CPU, IO and Memory to increase throughput.

Answer 5

Using match() against() you perform your research across your entire 3GB fulltext index and there is no way to force another index in this case.

To speed up your query you need to make your fulltext index lighter so you can:

1 - clean all the useless characters and stopwords from your fulltext index

2 - create multiple fulltext indexes and peek the appropriate one

3 - change fulltext searches to LIKE clause and force an other index such as 'id'.

Answer 6

Try placing id in the text index and say:

match(BODY,ID) against (+big +data +97) and id=97

You might also look at sphinx which can be used with MySQL easily.