TextFieldParser with not parsing last line

Question

I am importing .csv file in my sql database. I am using TextFieldParser.

My code is

TextFieldParser parser = new TextFieldParser(file); 
//single file 
//TextFieldParser parser = new TextFieldParser(CSVFolderPath + "\\" + file); 
parser.TextFieldType = FieldType.Delimited; 
parser.SetDelimiters(","); 
int k = 0; 
while (!parser.EndOfData) 
{ 
    //Processing row 
    string[] fields = parser.ReadFields(); 
    if (k != 0) 
    { 
        for (int i = 0; i < fields.Length; i++) 
        { 
            stationcode = fields[0].ToString().Substring(4, 5); 
            //if (fields[1].ToString().Substring(14, 8) == date) 
            //{ 
            if (i == 0) 
            { 

                dr = workTable.NewRow(); 
                dr[i] = fields[i].Substring(0, fields[i].Length - 4);  

            } 
            else if (i == 3) 
            { 
                dr[i] = Convert.ToDateTime(fields[i].ToString()); 
            } 
            else if (i == 4) 
            { 
                dr[i] = Convert.ToDateTime(fields[i].ToString()); 
            } 
            else if (i == 5) 
            { 
                dr[i] = Convert.ToInt32(fields[i].ToString()); 
            } 
            else 
                dr[i] = fields[i].ToString(); 
            if (i == 5) 
            { 
                workTable.Rows.Add(dr); 
            } 
            //} 

        } 
    } 
    k = k + 1; 
} 
parser.Close();     

Here worktable is DataTable.

Code parse file fine.

But in my csv file last line is used for summary. Total of some fields.

I dont want to include that line for inserting in datatable.

How can I do this ?

Answer 1

If memory space won't be an issue. Load the list of CSV lines first, and parse each line save the last. Otherwise you need some way of identifying summary rows. (such as null data for things like dates) which can signal the parser to ignore that row.

The best way to deal with data for import is to try and establish that import files will only include data to be imported. (No headers, no summaries, etc.) Unfortunately a lot of companies try importing Reports instead of actual export files.

Answer 2

If your last line start with some specific string ("Summary", for sample), you can use the CommentTokens property of TextFieldParser class.

See : http://msdn.microsoft.com/fr-fr/library/microsoft.visualbasic.fileio.textfieldparser.commenttokens(v=vs.110).aspx

And : http://geekswithblogs.net/brians/archive/2010/07/07/whats-a-nice-class-like-textfieldparser-doing-in-a-namespace.aspx

Answer 3

If you have many different cases required for identifying the summary row & memory space IS an issue, you can always use the TextFieldParser.PeekChars(Int32) method:

Reads the specified number of characters without advancing the cursor.

In the remarks section:

The numberOfChars value must be less than the total number of characters in the line. If it is not, the string returned by PeekChars will be truncated to the length of the line.

Therefor:

TextFieldParser parser = new TextFieldParser(file); 
//single file 
//TextFieldParser parser = new TextFieldParser(CSVFolderPath + "\\" + file); 
parser.TextFieldType = FieldType.Delimited; 
parser.SetDelimiters(","); 
int k = 0; 
while (!parser.EndOfData) 
{ 
    //Processing row 
    string[] fields = parser.ReadFields();

    //Peek if this is the last line, then break
    if (parser.PeekChars(Int32.MaxValue) is "") break;

    if (k != 0) 
    { 
...

docs.microsoft link