Remove comma inside Comma Delimited File csv in SSIS Using Script task

Question

I'm trying to load multiple csv files into a sql database using SSIS but I'm having some issue reading the csv file.

This is how the files are coming:

,,,,
ID, Name, Amount, Important_Dates, Company
101, Mark,"157,500",11/18/19, Amazon, Inc
102,Tom, "388,000",11/14/19, Ebay Corp
103,Tim, "484,000",11/25/19, Wish
104,Richard,"384,750",10/5/19, NBA. INC

Every time I try to open the file with SSIS the data get mixed because of the commas inside the values and this is how it looks:

And trying to read the file in a way that all values stay on its own column like:

Note: I'll be running this 2 or 1 times a day that's why I'm trying to automate it.

And I already tried the code from this page but didn't work: https://radacad.com/problem-with-comma-values-in-comma-delimited-file

Probably, a script task with C# code before the data flow, that will take care of the commas across all columns might help but I have no idea how to do that.

Any help or idea I'll really appreciate it. Thank you!

Answer 1

You should have a connection manager item for whatever the source of this data is, and the connection manager will allow you to set a text qualifier property. Do that, and SSIS should handle this data properly.

Answer 2

Your CSV is not so clean. I have edited and it's

ID,Name,Amount,Important_Dates,Company
101,Mark,"157,500",11/18/19,"Amazon,Inc"
102,Tom,"388,000",11/14/19,Ebay Corp
103,Tim,"484,000",11/25/19,Wish
104,Richard,"384,750",10/5/19,NBA. INC

Then you can change the separator using a classic command line tool as Miller ( https://github.com/johnkerl/miller ) and convert it from CSV to TSV, running

mlr --c2t cat input.csv >output.tsv

to have

ID      Name    Amount  Important_Dates Company
101     Mark    157,500 11/18/19        Amazon,Inc
102     Tom     388,000 11/14/19        Ebay Corp
103     Tim     484,000 11/25/19        Wish
104     Richard 384,750 10/5/19 NBA. INC

And then you should change import separator, choosing tab