可以并行处理非线程安全代码吗？

Question

I have a C# program that performs a foreach on DataRows . The simplified version is:

foreach (DataRow row in masterReportData.Rows)
{
    LoopThruReports(row, p2, p3, p4);
}

And LoopThruReports is declared as:

public static void LoopThruReports(DataRow row, DataTable p2, string p3, string p4)

I'd really like to have the iterations run in parallel because they fill a SqlDataAdapter from a database (which takes a while). Despite trying threads, tasks, and the parallel library, I cannot figure out if this is even possible to do. If only I could have each thread not share variables with other threads! Does anyone have any ideas on whether this is possible, and if so, how I could do it? Thanks.

EDIT: The LoopThruReports method calls GetGCHAReportDetailData , which is where the Fill is done. Here's what this method looks like:

    public static DataSet GetGCHAReportDetailData(int reinsuranceContract, int billingMode, DateTime reportPeriod)
    {
        DataSet dsResults = new DataSet();
        DataTable gchaReportData = new DataTable();

        string connStr = ConfigurationManager.ConnectionStrings["UtopiaConnString"].ConnectionString;

        using (SqlCommand sqlCmd = new SqlCommand("dbo.[sproc_GCHADetailDataForContract]"))
        {
            sqlCmd.CommandType = CommandType.StoredProcedure;

            //set the command time out to 60 minutes b/c this report takes time to generate
            sqlCmd.CommandTimeout = 0;

            SqlParameter parm = new SqlParameter("@ReinsuranceContractId", SqlDbType.Int);
            parm.Value = reinsuranceContract;
            sqlCmd.Parameters.Add(parm);

            SqlParameter parm1 = new SqlParameter("@BillingMode", SqlDbType.Int);
            parm1.Value = billingMode;
            sqlCmd.Parameters.Add(parm1);

            SqlParameter parm2 = new SqlParameter("@ReportingPeriod", SqlDbType.DateTime);
            parm2.Value = reportPeriod;
            sqlCmd.Parameters.Add(parm2);
            sqlCmd.CommandTimeout = 0;

            using (SqlConnection conn = new SqlConnection(connStr))
            {
                sqlCmd.Connection = conn;

                using (SqlDataAdapter da = new SqlDataAdapter(sqlCmd))
                {
                    try
                    {
                        conn.Open();
                        da.Fill(dsResults);

                        conn.Close();
                    }
                    catch (Exception ex)
                    {
                        logger.Error(ex, "Error retrieving data from the database.\r\n Message: {0}", ex.Message);
                        throw;
                    }
                    finally
                    {
                        if (conn.State != ConnectionState.Closed)
                            conn.Close();
                    }
                }
            }
        }

        return dsResults;
    }

Answer 1

Trying to multi-thread multiple Fill calls to the same SQL server will be a loosing battle. Both he SQL server and the network are already sending you data as fast as they can. By splitting the Fill calls over multiple threads you are actually going to slow things down. The bandwidth will be split equally between each thread (so there will be no performance increase), except each thread will also incur some overhead, which will slow things down a bit. Each thread you add will compound this problem more.

If each Fill runs against a different database then you might be able to get some threading improvement, as they won't be competing for the same SQL resources, though network contention will still be an issue.

I don't see anything in your code that would make multi-threading unsafe, I just don't see how threading it will improve your processing time.

---

Comments indicate that you are actually generating reports from the data, and that is what you want to multi-thread. This may actually work. As long as the reports only read data, and never write data, you should be able to thread them, and share a single dataset.

According to MSDN , DataTables and DataSet have this note "This type is safe for multithreaded read operations. You must synchronize any write operations."

It should be perfectly fine to share the same DataSet instance between threads as long as you only read from it, writing will require special handling.

Answer 2

This code seems to work:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ParallelDBProcessing
{
    class Program
    {
        static void Main(string[] args)
        {
            List<Task> taskList = new List<Task>();
            for (int i = 1; i <= 4; i++)
            {
                int temp = i;
                Task task = Task.Run(() => DoWork(temp));   // use Task.Run if you want to get task.Result back
                taskList.Add(task);
            }
            Task.WaitAll(taskList.ToArray());
        }

        public static void DoWork(int num)
        {
            Console.WriteLine(num);
            string connstr = @"Data Source = (local)\sqlexpress; Initial Catalog = Barry; Integrated Security = true";

            DataSet dsResults = new DataSet();
            using (SqlCommand sqlCmd = new SqlCommand("dbo.[spMagicProc]"))
            {
                sqlCmd.CommandType = CommandType.StoredProcedure;

                SqlParameter parm = new SqlParameter("@Input_A", SqlDbType.Int);
                parm.Value = num;
                sqlCmd.Parameters.Add(parm);

                SqlParameter parm1 = new SqlParameter("@Input_B", SqlDbType.Int);
                parm1.Value = 2;
                sqlCmd.Parameters.Add(parm1);

                using (SqlConnection conn = new SqlConnection(connstr))
                {
                    sqlCmd.Connection = conn;

                    using (SqlDataAdapter da = new SqlDataAdapter(sqlCmd))
                    {
                        try
                        {
                            conn.Open();
                            da.Fill(dsResults);
                            conn.Close();
                        }
                        catch (Exception ex)
                        {
                            throw;
                        }
                        finally
                        {
                            if (conn.State != ConnectionState.Closed)
                                conn.Close();
                        }
                    }

                }
                Console.WriteLine(dsResults.Tables[0].Rows.Count);
            }
        }
    }
}