Delete consecutive duplicates from a table in mysql

Question

I have a database , where table contains consecutive duplicate rows . Demo of table with data is as follows.

id  name  processed
 1  xyz      0
 2  xyz      0
 3  ABC      0
 4  ABC      0

I want to delete the consecutive duplicate from this table , and once duplicate is deleted update processed to 1. So that the final table looks like follows.

id  name  processed
 1  xyz      1
 3  ABC      1

I am doing it as follow.

SET @v1 := (select group_concat(`id`) from `names` as m1 where 0 < (select count(*) from `names` as m2 where m2.`id` = m1.`id` - 1 and m2.`name` = m1.`name`));
DELETE FROM names WHERE id IN (@v1);
UPDATE names SET `processed`=1 WHERE `processed`=0

The query works fine , but it deletes one row at a time . Please help me on this.I want all the selected rows to be deleted . Thanks in advance.

Answer 1

As @MarkBaker already wrote in comment, you can try DELETE FROM table WHERE name=name and id>id .

But that's only fix to what already has been done. To prevent that, you should add unique index to name column. That should prevent any duplicates of being added in future.

You can't set unique index when you have duplicates though, so you need clean first :)

Answer 2

You cannot UPDATE and DELETE in the same query. So that pretty much leaves you with this:

DROP TABLE IF EXISTS my_table;
CREATE  TABLE my_table 
(id  INT NOT NULL AUTO_INCREMENT PRIMARY KEY,name  CHAR(3) NOT NULL,processed TINYINT NOT NULL DEFAULT 0);

INSERT INTO my_table VALUES
(1  ,'xyz',      0),
(2  ,'xyz',      0),
(3  ,'ABC',      0),
(4  ,'ABC',      0);

SELECT * FROM my_Table;
+----+------+-----------+
| id | name | processed |
+----+------+-----------+
|  1 | xyz  |         0 |
|  2 | xyz  |         0 |
|  3 | ABC  |         0 |
|  4 | ABC  |         0 |
+----+------+-----------+

SELECT y.* FROM my_table x JOIN my_table y ON y.id = x.id + 1 AND y.name = x.name;
+----+------+-----------+
| id | name | processed |
+----+------+-----------+
|  2 | xyz  |         0 |
|  4 | ABC  |         0 |
+----+------+-----------+

DELETE y FROM my_table x JOIN my_table y ON y.id = x.id + 1 AND y.name = x.name;
Query OK, 2 rows affected (0.00 sec)


UPDATE my_table SET processed = 1;
Query OK, 2 rows affected (0.00 sec)

SELECT * FROM my_table;
+----+------+-----------+
| id | name | processed |
+----+------+-----------+
|  1 | xyz  |         1 |
|  3 | ABC  |         1 |
+----+------+-----------+

Answer 3

For PHP and MySQL , If your all data is consecutive pairs then this will work.

$con = mysqli_connect('host', 'user', 'pass', 'db');
$query ="select m1.id from names as m1 where 0 < (select count(*) from names as m2 where m2.id = m1.id - 1 and m2.name = m1.name)";
$result = mysqli_query($con, $query);
while($row = mysqli_fetch_array($result)){
  $query2 ="DELETE FROM names WHERE id = ".$row['id'];
  mysqli_query($con, $query2);
  $id = $row['id']-1;
  $query3 ="UPDATE names SET `processed`=1 WHERE id = ".$id;
  mysqli_query($con, $query3);
}

I checked it and its working fine. Hope it works for you too.

Answer 4

This will not edit your table, but will give you SELECT with desired result:

SELECT min(id) id, name, 1 processed
FROM mytable
GROUP BY name

You can use this in CREATE TABLE newtable AS SELECT ... , and then DROP mytable , and finally to ALTER TABLE newtable RENAME TO mytable .

Answer 5

The DISTINCT keyword can be used to return only distinct (different) values. Use this query:

SELECT DISTINCT `id`, `name`,`1` AS processed
FROM mytable;