mysqlimport keeps missing rows

Question

I'm using mysqlimport to ingest a csv file.

This is the command that I used:

mysqlimport --ignore-lines=1 --fields-terminated-by=, --columns=$COLUMNS  --lines-terminated-by="\n" \
        --verbose --local --fields-enclosed-by='"' --host=$HOST --port=$PORT \
        --user=$USERNAME --password=$PASSWORD ${DATABASE} $FILE_PATH

Before I started the process, I truncated the table and it does not have any unique indexes.

The source file has 7288362 records(At beginning I thought the csv rows may contain new line characters. I did a check but there was no quoted new line characters in the fileds).

However when mysqlimport was done, it said "Records: 7288300 Deleted: 0 Skipped: 0 Warnings: 43730142".

When I logged into mysql and did a select count(*) from MyTable, it also gave me 7288300.

Any ideas?

Thanks!

Answer 1

I have this kind of rows in my csv files:

1345235,62,"Theme From \Shaft\""""",,Chris Davis & Kim Waters,,,,,,5,0,,,,,,,,,,2016-10-28,,,,2016-10-28,,Y,,,,,,0,‌​2016-10-28 09:09:39

The multi quotes there caused a problem. When I added --fields-escaped-by='' , it worked.