Threading and Incrementing

Question

So I think this might be a fundamental flaw in my approach to threading and incrementing a global counter, but here is my problem. I have a collection of file names from a DB that I iterate through, and for each file name I search it within a top level folder. Each iteration I thread the search and increment a counter when it completes so I can determine when it finishes. The problem is the counter never ever ever gets as high as the total file count, it comes very close sometimes, but never reaches what I would expect.

public class FindRogeRecords
{
    private delegate void FindFileCaller(string folder, int uploadedID, string filename);
    private Dictionary<int, string> _files;
    private List<int> _uploadedIDs;
    private int _filesProcessedCounter;
    private bool _completed;

    public void Run()
    {
        _files = GetFilesFromDB(); //returns a dictionary of id's and filenames
        FindRogueRecords();
    }

    private void FindRogueRecords()
    {
            _uploadedIDs = new List<int>();

        foreach (KeyValuePair<int, string> pair in _files)
        {
            var caller = new FindFileCaller(FindFile);
            var result = caller.BeginInvoke(txtSource.Text, pair.Key, pair.Value, new AsyncCallback(FindFile_Completed), null);
        }
    }

    private void FindFile(string documentsFolder, int uploadedID, string filename)
    {
        var docFolders = AppSettings.DocumentFolders;

        foreach (string folder in docFolders)
        {
            string path = Path.Combine(documentsFolder, folder);
            var directory = new DirectoryInfo(path);
            var files = directory.GetFiles(filename, SearchOption.AllDirectories);

            if (files != null && files.Length > 0) return;
        }

        lock (_uploadedIDs) _uploadedIDs.Add(uploadedID);
    }

    private void FindFile_Completed(System.IAsyncResult ar)
    {
        var result = (AsyncResult)ar;
        var caller = (FindFileCaller)result.AsyncDelegate;

        _filesProcessedCounter++;
        _completed = (_files.Count == _filesProcessedCounter); //this never evaluates to true
    }
}

Answer 1

You are accessing _filesProcessedCounter variable from multiple threads without any synchronization (even simple lock() ) so this caused Race Condition in your code.

To increment an integer variable you can use Interlocked.Increment() which is thread-safe but considering that following line of code requires synchronization as well:

_completed = (_files.Count == _filesProcessedCounter); 

I would suggest using lock object to cover both lines and keep code much clear:

// Add this field to a class fields list
private static readonly object counterLock = new object();

// wrap access to shared variables by lock as shown below
lock (counterLock)
{
  _filesProcessedCounter++;
  _completed = (_files.Count == _filesProcessedCounter);   
}

Answer 2

This is because you have a race condition in your program. Since the ++ operator is equal to the following code

c = c + 1; // c++;

your can see, that it is not atomic. A thread that increments the value stores c in a register, increments the value by 1 and then writes it back. When the thread now gets pushed aside because another thread gets the CPU the execution of c = c + 1 may not be finished. A second thread performs the same action, reads the old c and increments its value by one. When the first thread gains the CPU again he would overwrite the data written by the second thread.

You can use locks to make sure, that only one thread can access the variable at one time, or use atomic functions like

Interlocked.Increment(ref c); 

to increase c in a threadsave manner;

Answer 3

使用Interlocked.Increment(ref _filesProcessedCounter) 。

Answer 4

i think it should be:

if (files != null && files.Length > 0) continue; 

and not return (you will not increase your counter...)

Answer 5

Convert

_filesProcessedCounter++;
_completed = (_files.Count == _filesProcessedCounter);

to

_completed = (_files.Count == Interlocked.Increment(ref _filesProcessedCounter));