How to synchronize threads in socket server for writing to file

Question

I am writing a socket server which receives many client request to write to a file. I have an arraylist to store the socketlisteners. After I ran the client to send up to 25 requests, which results in the socket server storing 25 socketlisteners and starting 25 threads to receive the data and write to the file. After running I get errors from the server that its trying to access the same file which is locked by another process. I also get null reference errors. So, what is the best way to synchronize the threads so all the requests get processed and all the data get written to the file.

Any help is appreciated. Thanks. -CR

string data = Encoding.ASCII.GetString(byteBuffer,0, size);

if (!File.Exists(filepath))
{
    using (StreamWriter sw = File.CreateText(filepath))
    {
        sw.WriteLine(data);  
    }
}
else
{
    using (StreamWriter sw = File.AppendText(filepath))
    {
        sw.WriteLine(data);
    }
}

Answer 1

This is a classical Producer Consumer problem , you can easily solve it by implementing Pipeline pattern.

Basically all threads would write into the same thread-safe queue rather than file. And you need one more thread which will read from the shared collection all available requests one by one and write to a file.

Using .NET 4.X it is easy to implement using BlockingCollection class which is thread-safe collection.

// request queue represented by a single string
var requestsQueue = new BlockingCollection<string>(); 

Request processing logic (which run in multiple threads) basically just add requests into the queue:

requestsQueue.Add(request);

Thread/Task which dumps requests to a file should consume available requests in following way:

using (StreamWriter sw = File.CreateText(filepath))     
{         
    foreach (var request in requestsQueue.GetConsumingEnumerable())
    {
       sw.WriteLinerequest);
    }
}

Answer 2

In such situations it's better to serialize access to the file and forward all write requests to a separate thread via a synchronized queue for two main reasons:

1. Writing/reading to a single file on a mechanical disk from multiple threads will not yield any performance benefit (if there are many threads you will even see performance degradation).
2. You will avoid a lot of synchronization problems, especially if you use one of the predefined synchronized queue like ConcurrentQueue .