Why is C# List<T> Add method very slow?

Question

I have an ASP.NET MVC project using Dapper to read data from a database and I need to export to Excel.

Dapper is fast! ExecuteReader takes only 35 seconds.

But list.Add(InStock); spends too much time! Over 1020 seconds!

Do you have any idea why this is?

public List<InStock> GetList(string stSeId, string edSeId, string stSeDay, string edSeDay, string qDate)
{
    List<InStock> list = new List<InStock>();
    InStock InStock = null;

    IDataReader reader;

    using (var conn = _connection.GetConnection())
    {
        try
        {
            conn.Open();
            //******************Only 35 seconds*****
            reader = conn.ExecuteReader(fileHelper.GetScriptFromFile("GetInStock"),
                 new { STSeId = stSeId, EDSeId = edSeId, STSeDay = stSeDay, EDSeDay = edSeDay, qDate = qDate });
            //*************************************

            //******************Over 1020 seconds**********
            while (reader.Read())
            {
                InStock = new InStock();

                InStock.ColA = reader.GetString(reader.GetOrdinal("ColA"));
                InStock.ColB = reader.GetString(reader.GetOrdinal("ColB"));
                InStock.ColC = reader.GetString(reader.GetOrdinal("ColC"));

                list.Add(InStock);
            }
            //*********************************************
            return list;
        }
        catch (Exception err)
        {
            throw err;
        }
    }
}

Answer 1

It's the database.

From Retrieve data using a DataReader ,

The DataReader is a good choice when you're retrieving large amounts of data because the data is not cached in memory.

The key clue for your performance concern regards "because the data is not cached in memory". While strictly an implementation detail, each call to Read() gets new data from the database, while the List<InStock>.Add() call is just adding the new InStock to the list.

There are orders of magnitude of difference in processing times between disk access (even SSDs) compared to RAM. And theres orders of magnitude of difference between network requests and disk access. There's not really a conceivable way that anything other than the database access is the cause of most of your run time.

--

As a side note, you're going to exceed the maximum number of rows in an Excel worksheet.