SQL export to CSV file using .Net

Question

I have an app that saves a sql data set to a csv file. I read the data, loop each row saving into a string and then I save the string into a text file. The problem is there is a over 25k rows and it takes a long time to process. Is there a faster way to do this?

SqlDataAdapter sSQL2 = new SqlDataAdapter("SelUsersEmails", Conn);
DataSet RS2 = new DataSet();
sSQL2.Fill(RS2, "SelUsersEmails");


s = "UserID,SignUpDate,Email,PayPalEmail,Firstname,Lastname" + Environment.NewLine;



foreach (DataRow u in RS2.Tables["SelUsersEmails"].Rows)
{//loop each
        s += u["UserID"].ToString() + "," + u["SignUpDate"].ToString() + "," +["Email"].ToString() + "," + u["NetworkEmail"].ToString() + "," + ["Firstname"].ToString() + "," + u["Lastname"].ToString();

        s += Environment.NewLine;

}

Answer 1

There are many things that can be done - instead of string concatenation you can use StringBuilder to create it more efficiently.

Another option it to stream the data to the file as you go along - this is the most memory efficient way, though can be more IO intensive.

But I suggest using FileHelpers to create the CSV file. It is a popular library that is guaranteed to implement CSV properly (for example - what happens with your code if a field contains a comma?).

Answer 2

Why don't you open the file ready for writing and write each line as you're looping through the resultset?

That's the most efficient way of creating the CSV file.