StringBuillder.Tostring() throws out of memory Exception

Question

I am getting out of memory exception even after i cleared the stringbuilder object.

I want to convert a data table to CSV, when i am trying to convert it into string i am getting the exception.

The code is below:

        int lineCount = 0;
        const int capacity = 10000000; 
        StringBuilder csvBuilder = new StringBuilder(capacity);
        if (dt != null && dt.Rows.Count > 0)
        {
                    using (var sw = new StreamWriter(FileName, false))
                    {
                        foreach (DataRow dr in dt.Rows)
                        {
                            csvBuilder.AppendLine(String.Join(fileSeperator, dr.ItemArray));
                            lineCount++;
                            if (lineCount >= 100000)
                            {

                                sw.Write(csvBuilder.ToString());
                                csvBuilder.clear();
                                lineCount = 0;
                            }                              

                        }


                        sw.Write(csvBuilder.ToString());
                        csvBuilder.clear();
                        lineCount = 0;
                    }

            }

Answer 1

I am getting out of memory exception even after i cleared the stringbuilder object.

Yes, that looks plausible. You're not running out of memory as such but you are suffering from fragmented memory. The SB will be placed on the Larg Object heap and the LOH is collected but not compacted. Clearing the StringBuilder won't help.

The solution is of course simple: use a smaller buffer, or non at all.

//const int capacity = 10000000; 
  const int capacity = 1000000;    // depends on average linelength

//if (lineCount >= 100000)
  if (lineCount >= 1000)

You seem to use a StringBuilder to buffer I/O, did you profile that thoroughly? The FileStream will already buffer, and when you want to help you should aim at a buffer from 8 - 16 kB.

Answer 2

you can check the length

if (csvBuilder.Length >= 100000)
{

    sw.Write(csvBuilder.ToString());
    csvBuilder.clear();
    lineCount = 0;
}
csvBuilder.AppendLine(String.Join(fileSeperator, dr.ItemArray));
lineCount++;