如何一次遍历100个记录的数据集？ C＃

Question

I have a dataset that was roughly 4500 rows. I was passing it to a webservice but after performance issues and timeouts we want to be able to pass 100 of those rows at a time.

How can this be achieved in c#?

Do I need to split the dataset into tables of 100 and then loop through the tables or is there an easier way to send sets of 100 rows at a time?

Answer 1

You could use Linq, specifically Take and the CopyToDataTable extension.

Example:

    DataSet ds = new DataSet();
    DataTable dt = ds.Tables["YOURDATATABLE"];
    IEnumerable<DataRow> firstHundred = dt.AsEnumerable().Take(100);
    // create datatable from query
    DataTable boundTable = firstHundred.CopyToDataTable<DataRow>();
    //call your web service with 1st hundred
    //
    IEnumerable<DataRow> nextHundred = dt.AsEnumerable().Skip(100).Take(100);
    // and so on
    boundTable = nextHundred.CopyToDataTable<DataRow>();
    //call your web service with 1st hundred
    //

Example with simple for and Linq which takes into account you have 4,500 rows and want to chunk it by groups of 100:

    DataSet ds = new DataSet();
    DataTable dt = ds.Tables["YOURDATATABLE"];
    IEnumerable<DataRow> firstHundred = dt.AsEnumerable().Take(100);
    // create datatable from query
    DataTable boundTable = firstHundred.CopyToDataTable<DataRow>();
    //call your web service with 1st hundred
    //

    int skipCount = 0;

    for (int i = 1; i <= 45; i++)
    {
        skipCount += 100;
        IEnumerable<DataRow> nextHundred = dt.AsEnumerable().Skip(skipCount).Take(100);
        // create datatable from query
        DataTable boundTable = nextHundred.CopyToDataTable<DataRow>();
        // call web service with next hundred and so on
    }

Answer 2

private IEnumerable<IEnumerable<DataRow>> GetChunks(DataTable table, int size)
{
    List<DataRow> chunk = new List<DataRow>(size);

    foreach (DataRow row in table.Rows)
    {
        chunk.Add(row);
        if (chunk.Count == size)
        {
            yield return chunk;
            chunk = new List<DataRow>(size);
        }
    }

    if(chunk.Any()) yield return chunk;
}

//....

DataTable table = dataSet.Tables[yourTable];
var chunks = GetChunks(table, 100);

foreach (IEnumerable<DataRow> chunk in chunks)
    SendChunk(chunk); // <-- Send your chunk of DataRow to webservice

You can also attempt to send the data on parallel:

// This would replace the above last two lines
Parallel.ForEach(chunks, chunk => SendChunk(chunk));

Although I'd not recomment that, since SendChunk is an I/O operation.

Instead, try to convert your code to async , and you can get better results:

// Will execute all tasks at the same time
// SendChunk should return a Task
await Task.WhenAll(chunks.Select(chunk => SendChunk(chunk)).ToArray());