Mysql delete and optimize very slow

Question

I searched Internet and Stack Overflow for my trouble, but couldn't find a good solution.

I have a table (MySql MyISAM) containing 300,000 rows (one column is blob field).

I must use:

DELETE FROM tablename WHERE id IN (1,4,7,88,568,.......)

There are nearly 30,000 id's in the IN syntax.

It takes nearly 1 hour. Also It does not make the .MYD file smaller although I delete 10% of it, so I run OPTIMIZE TABLE... command. It also lasts long...(I should use it, because disk space matters for me).

What's a way to improve performance when deleting the data as above and recover space? (Increasing buffer size? which one? or else?)

Answer 1

With IN , MySQL will scan all the rows in the table and match the record against the IN clause. The list of IN predicates will be sorted, and all 300,000 rows in the database will get a binary search against 30,000 ids.

If you do this with JOIN on a temporary table (no indexes on a temp table), assuming id is indexed, the database will do 30,000 binary lookups on a 300,000 record index.

So, 300,000 binary searches against 30,000 records, or 30,000 binary searches against 300,000 records... which is faster? The second one is faster, by far.

Also, delaying the index rebuilding with DELETE QUICK will result in much faster deletes. All records will simply be marked deleted, both in the data file and in the index, and the index will not be rebuilt.

Then, to recover space and rebuild the indexes at a later time, run OPTIMIZE TABLE .

Answer 2

The size of the list in your IN() statement may be the cause. You could add the IDs to a temporary table and join to do the deletes. Also, as you are using MyISAM you can use the DELETE QUICK option to avoid the index hit whilst deleting:

For MyISAM tables, if you use the QUICK keyword, the storage engine does not merge index leaves during delete, which may speed up some kinds of delete operations.

Answer 3

try this

1. create a table name temptable with a single column id
2. insert into table 1,4,7,88,568,......
3. use delete join something like
4. DELETE ab, b FROM originaltable AS a INNER JOIN temptable AS b ON a.id= b.id where b.id is null;

its just an idea . the query is not tested . you can check the syntax on google.

Answer 4

I think the best approach to make it faster is to create a new table and insert into it the rows which you dont want to delete and then drop the original table and then you can copy the content from the table to the main table.

Something like this:

INSERT INTO NewTable SELECT * FROM My_Table WHERE ... ;

Then you can use RENAME TABLE to rename the copy to the original name

RENAME TABLE My_Table TO My_Table_old, NewTable TO My_Table ;

And then finally drop the original table

DROP TABLE My_Table_old;