批量仅插入新行

Question

I have aa group of csv files that I'm using to populate a SQL database. I'm setting this up to be a daily process. I've just discovered though that a handful of the files come in each day with all the historical data, not just the daily updates.

When I try to do the bulk insert this causes an error because the primary key is being violated.

I thought that setting IGNORE_DUP_KEY = ON would stop the non-unique records from being inserted.

CREATE TABLE TABLE1
(
Column1 varchar(32),
Column2 varchar(32),
Column3 char(9),
Column4 int,
Column5 float(53),
Column6 date,
CONSTRAINT pk_One
    PRIMARY KEY (Column1, Column2, Column6)
    WITH (IGNORE_DUP_KEY = ON)
);

Then when I try to run the script that updates the tables I get the unhelpful error message "Associated statement is not prepared (0)."

I could get around this by writing the results into a separate table, then writing the new rows into the table proper, but having separate handling for the different tables strikes me as painful, and ugly.

Is there an easy way to just tell SQL to only write the rows that don't violate the primary key constraint?

Answer 1

Insert into table1 (column1) value (value1)

... So, if you just want to ignore inserts that fail, do INSERT IGNORE INTO. Then it'll try to insert the values, fail, and move on gracefully. If you want it to replace the duplicate key values with the new values it attempted to insert, check out : (below, ignore optional, depends on if you want it to throw errors.)

INSERT [IGNORE] INTO table1 (column1) values (value1) ON DUPLICATE KEY UPDATE

so "insert ignore" is the answer, on inserts not on table creation. refer to mysql docs . and ON DUPLICATE KEY UPDATE is useful to know.