Mysql Query to delete duplicates

Question

I have duplicate results like below where some column may have data and may not

| contact_info | icon | id  | title         | lastmodified_by  |
+--------------+------+-----+---------------+------------------+
|          169 |  305 | 123 | Whakarewarewa | 2011100400305262 |
|         NULL | NULL | 850 | Whakarewarewa | NULL             |
+--------------+------+-----+---------------+----------------



| contact_info | icon | id  | title         | lastmodified_by  |
+--------------+------+-----+---------------+------------------+
|         NULL | NULL | 123 | Paris         | NULL             |
|         NULL | NULL | 850 | Paris         | NULL             |
+--------------+------+-----+---------------+----------------

I want to delete record having less Data and if the all the Field values are exact same then delete any row. There are thousand records like this.

Answer 1

Try this two-step solution:

Run this query to vew all duplicates - record having less Data -

SELECT t1.* FROM table t1
  JOIN (
    SELECT
      title,
      MIN(IF(contact_info IS NULL, 0, 1) + IF(contact_info IS NULL, 0, 1) + IF(lastmodified_by IS NULL, 0, 1)) min_value_data,
      MAX(IF(contact_info IS NULL, 0, 1) + IF(contact_info IS NULL, 0, 1) + IF(lastmodified_by IS NULL, 0, 1)) max_value_data
    FROM table GROUP BY title HAVING min_value_data <> max_value_data
  ) t2
  ON t1.title = t2.title AND IF(t1.contact_info IS NULL, 0, 1) + IF(t1.contact_info IS NULL, 0, 1) + IF(t1.lastmodified_by IS NULL, 0, 1) <> t2.max_value_data

Rewrite it to DELETE statement and execute.

---

Then run this query to remove all duplicates except min ID:

DELETE t1 FROM table t1
  JOIN (SELECT MIN(id) id, title FROM table GROUP BY title) t2
    ON t1.id <> t2.id AND t1.title = t2.title;

Answer 2

Use this to select duplicates, feel free to alter this to a delete statement:

SELECT * FROM `test`,
(SELECT title, count( title ) AS ttl
FROM `test`
GROUP BY title
HAVING ttl >1) AS sub

WHERE test.title = sub.title
AND contact_info IS NULL AND lastmodified_by IS NULL 

Answer 3

Main table = tes1

Create temp

CREATE TEMPORARY TABLE my_temp ( id INT(20) NOT NULL ) ENGINE=MEMORY;

Fill with id's to remove

INSERT INTO my_temp (id) SELECT id FROM tes1 AS main, ( SELECT title, count( title ) AS ttl FROM tes1 GROUP BY title HAVING ttl >1 ) AS sub WHERE main.title = sub.title AND main.contact_info IS NULL AND main.lastmodified_by IS NULL GROUP BY main.contact_info, main.icon, main.title, main.lastmodified_by;

Delete!

DELETE FROM tes1 WHERE id IN (select id from my_temp);

Cleanup, note: do we really need this?

DROP TABLE my_temp;