SAP Data Services .csv data file load from Excel with special characters

Question

I am trying to load data from an Excel .csv file to a flat file format to use as a datasource in a Data Services job data flow which then transfers the data to an SQL-Server (2012) database table.

I consistently lose 1 in 6 records.

I have tried various parameter values in the file format definition and settled on setting Adaptable file scheme to "Yes", file type "delimited", column delimeter "comma", row delimeter {windows new line}, Text delimeter ", language eng(English) and all else as defaults.

I have also set "write errors to file" to "yes" but it just creates an empty error file (I expected the 6,000 odd unloaded rows to be in here).

If we strip out three of the columns containing special characters (visible in XL) it loads a treat so I think these characters are the problem.

The thing is, we need the data in those columns and unfortunately, this .csv file is as good a data source as we are likely to get and it is always likely to contain special characters in these three columns so we need to be able to read it in if possible.

Should I try to specifically strip the columns in the Query source component of the dataflow? Am I missing a data-cleansing trick in the query or file format definition?

Answer 1

好的，所以没有得到我想要的答案，但是通过将“文本字符串中的行”参数设置为“行定界符”使它起作用。