Import CSV to SQL using schema.ini as validator

Question

I'm using schema.ini to validate the data types/columns in my CSV file before loading into SQL. If there is a datatype mismatch in a row, it will still import the row but leaves that particular mismatch cell blank. Is there a way in which I can stop user from importing the CSV file if there is any issues and/or provide a error report (ie which row has problems).

Answer 1

The best approach is to check the file for any mismatch; but in the case of a large file, then this is not feasible. You might need to load it first the check the loaded data in the table for the mismatch. This is much faster than checking the file (You can use a simple T-SQL script to check for nulls in the table). IF mismatches are found, the user can then be notified and the table can then be cleared.

Answer 2

have a look at he FileHelpers library: http://www.filehelpers.com/

This is a very powerful library to do all kinds of imports, including csv and they also have a pretty neat error handling part

Using the Differents Error Modes The FileHelpers library has support for 3 kinds of error handling.

In the standard mode you can catch the exceptions when something fail. This approach not is bad but you lose some info about the current record and you can't use the records array because is not asigned.

A more intelligent way is usign the ErrorMode.SaveAndContinue of the ErrorManager:

Using the engine like this you have the good records in the records array and in the ErrorManager you have te records with errors and can do wherever you want.

Another option is to ignore the errors and continue how is showed in this example

1 engine.ErrorManager.ErrorMode = ErrorMode.IgnoreAndContinue; 2 3 records = engine.ReadFile(... Copy to Clipboard | View Plain | Print | ? engine.ErrorManager.ErrorMode = ErrorMode.IgnoreAndContinue;

records = engine.ReadFile(... In the records array you only have the good records.