How to remove Double Quotes from a CSV File insertion into an SQL DB

Question

I'm receiving multiple CSV files but some of them have every value inside a double quoting, which breaks every further processing inside SQL.

The application already has a method that takes some of the CSV Files (based on their name), reads them line by line and appends a value to the end of the String to create a new column, and then, writing to a new file, without replacing the old one.

I thought about re-using that method and applying it to every file, but instead of appending new values to each line, removing the " characters, this should work just fine, but since some of the files are really big (Some of them around 1GB) i'd like to avoid unnecessary performance issues.

Is there a better way to replace, delete or ignore every " character, either in Java or during/after the SQL insertion?

Answer 1

You can simply REPLACE the Double Quotes (") from your table using the following Query.

UPDATE TableName
SET Column1 = REPLACE(Column1,'"',''),
Column2 = REPLACE(Column2,'"','')
...
...
...
ColumnN = REPLACE(ColumnN,'"','')