纠正 mysql 中不同拼写列的最佳方法

Question

I have a table with column that has data with spelling errors. Like: apple, appl, aple bana, banana, banna cat, cot, cta

I would like to correct all error spelling to single correct ones. There are thousands of rows. What would be the best approach to correct this issue where I wouldn't have to update each spelling errors manually? I have added status iscorrect 'Y' for correct ones.

Answer 1

Here's a thought, using SOUNDEX. SOUNDEX is really a lousy function, and certainly no panacea, but it might reduce a data set comprising thousands of errors to a data set comprising hundreds of errors.

For the rest, we can look at things like Levenshtein distance, but ultimately, you're going to need a manual approach to some extent...

DROP TABLE IF EXISTS bad_data;

CREATE TABLE bad_data
(id SERIAL PRIMARY KEY
,string VARCHAR(12) NOT NULL
);

INSERT INTO bad_data (string) VALUES
('apple'), 
('appl'), 
('aple'),
('bana'),
('banana'), 
('banna'),
('cat'),
('cot'), 
('cta');

DROP TABLE IF EXISTS good_data;

CREATE TABLE good_data
(id SERIAL PRIMARY KEY
,string VARCHAR(12) NOT NULL UNIQUE
);

INSERT INTO good_data(string) VALUES
('apple'), 
('banana'), 
('cat');

SELECT * 
  FROM bad_data x 
  JOIN good_data y ON SOUNDEX(x.string) = SOUNDEX(y.string);
+----+--------+------+--------+
| id | string | id   | string |
+----+--------+------+--------+
|  1 | apple  |    1 | apple  |
|  2 | appl   |    1 | apple  |
|  3 | aple   |    1 | apple  |
|  4 | bana   |    2 | banana |
|  5 | banana |    2 | banana |
|  6 | banna  |    2 | banana |
|  7 | cat    |    3 | cat    |
|  8 | cot    |    3 | cat    |
|  9 | cta    |    3 | cat    |
+----+--------+------+--------+