根据行数将数据集拆分为多个表

Question

I have a situation where I need to Split dataset results in to multiple tables eventually in to an array based on number of rows. Ex: my dataset has 34 rows with a url column, I need to split 34 rows in to 4 data tables (10,10,10 remaining 4) and eventually add to an array. I am using a forms windows application. Itried something like below however every time i add records to array it adds entire dataset. Any help would be appreciated.

 private DataSet Process(DataSet ds)
        {


            string[] Array1 = new string[10];
            string[] Array2 = new string[10];
            string[] Array3 = new string[10];
            string[] Array4 = new string[10];


            int COunt = ds.Tables[0].DefaultView.Count;
            int NoOfArraysToCreate = COunt/10 + 1;

            for (int i = 0; i <= NoOfArraysToCreate; i++ )
            {
                if (i == 0)
                {
                    foreach (DataRow drs in ds.Tables[0].Rows)
                    {
                        List<String> myList = new List<string>();
                        myList.Add(drs["Url"].ToString());
                        Array1 = myList.ToArray();
                    }
                }
                else if (i == 1)
                {
                    foreach (DataRow drs in ds.Tables[0].Rows)
                    {
                        List<String> myList = new List<string>();
                        myList.Add(drs["Url"].ToString());
                        Array2 = myList.ToArray();
                    }
                }
                else if (i == 2)
                {

                    foreach (DataRow drs in ds.Tables[0].Rows)
                    {
                        List<String> myList = new List<string>();
                        myList.Add(drs["Url"].ToString());
                        Array3 = myList.ToArray();
                    }
                }
                else if (i == 3)
                {

                    foreach (DataRow drs in dsURLsList.Tables[0].Rows)
                    {
                        List<String> myList = new List<string>();
                        myList.Add(drs["Url"].ToString());
                        Array4 = myList.ToArray();
                    }
                }

}

Answer 1

It seems you are looping through all DataTable rows for each of your pages. I would suggest you only loop over your rows once:

private List<List<string>> Process(DataSet ds, int pageSize)
{
    List<List<string>> result = new List<List<string>>();

    int COunt = ds.Tables[0].DefaultView.Count;
    int NoOfArraysToCreate = COunt / pageSize + 1;

    IEnumerable<DataRow> collection = ds.Tables[0].Rows.Cast<DataRow>(); //I find it easier to work with enumerables as it allows for LINQ expressions as below

    for (int i = 0; i < NoOfArraysToCreate; i++)
    {
        result.Add(collection.Skip(i*pageSize)
                .Take(pageSize)
                .Select(r => r["Url"].ToString())
                .ToList());
    }

    Parallel.ForEach(result, (page) =>
    {
        Parallel.ForEach(page, (url) => {}); // process your strings in parallel
    });

    return result;//I see you convert your string arrays back to DataSet, but since I don't know the table definition, I'm just returning the lists
}
void Main()
{
    // this is just a test code to illustrate my point, yours will be different
    var ds = new DataSet();
    var dt = new DataTable();
    dt.Columns.Add("Url", typeof(string));
    for (int i = 0; i < 34; i++) {
        dt.Rows.Add(Guid.NewGuid().ToString());//generating some random strings, ignore me
    }
    ds.Tables.Add(dt);
    //---------------------------------------
    Process(ds, 10);// calling your method
}

of course there are ways to do it with for loops as well, but I'd leave that for you to explore. I would also say hardcoding table numbers into your method usually is considered a code smell, but since I don't know your context I will not make any further changes