How to call the GetOrAdd method on a .NET ConcurrentDictionary in a thread-safe way?

Question

I'm trying to have a collection of keys which .. if the key doesn't exist, then I need to DoSomeMethod() and then add the key to the collection.

Problem is, this needs to be able to handle multiple threads trying to add the same key, at the same time.

If two threads have the same key, then only one will do DoSomeMethod() while the other needs to wait.

I've looked at using a ConcurrentDictionary and GetOrAdd ( with the Func(..) param option ) method but that seems to both 'fire off' at the same time if the two threads have the same key. I thought that the implementation of GetOrAdd would be

• lock 'key'
• get value from key .
• if no value then do whatever .. and now set value.
• return value .
  ... and any other key hits will wait until the lock is done.

It feels like my custom method which the GetOrAdd method calls, isn't thread safe.

The MSDN docs also suggest this?

Remarks
If you call GetOrAdd simultaneously on different threads, addValueFactory may be called multiple times, but its key/value pair might not be added to the dictionary for every call

---

Contrite example: I'm copy files from source to destination .

• When copying a source file, check the collection if we've tried to check-and-create the destination folder.
• If the collection doesn't have the key then check if the destination folder(s) do not exist ... and create it if it doesn't exist.
• Once the destination folder is created, store this folder name/path in the collection.
• Repeat for all files.

So in effect, we only create destination folders if we haven't already done it.

It's like i'm wanting to lock on a collection KEY ....

Answer 1

public class OnceOnlyConcurrent<TKey, TValue>
{
    private readonly ConcurrentDictionary<TKey, Lazy<TValue>> _dictionary = new ConcurrentDictionary<TKey, Lazy<TValue>>();

    public TValue GetOrAdd(TKey key, Func<TValue> computation)
    {
        var result = _dictionary.AddOrUpdate(key, _ => new Lazy<TValue>(computation, LazyThreadSafetyMode.ExecutionAndPublication), (_, v) => v);
        return result.Value;
    }
}

I guess I should describe this a bit. Basically what happens here is that while AddOrUpdate will always call the addValueFactory delegate twice if two callers happen upon AddOrUpdate at the same time, both of these calls don't really do anything but return a Lazy<T> reference that wraps the computation.

Inside AddOrUpdate , both results will be captured, but one will be dropped. Only a single instance of Lazy<T> will be returned to both callers of AddOrUpdate , so a single Lazy<T> will govern the computation being called.

Then, on the next line, when we ask for .Value , that will actually trigger the computation on one of the callers of this custom GetOrAdd and the other will block while the first computes - this is the functionality of the second argument to Lazy<T> ( LazyThreadSafteMode.ExecutionAndPublication ). BTW, this is the default behavior of Lazy<T> so you don't really need the second argument - I just used it to be more clear in this post.

Of course, this code could also be written as an extension method, but unfortunately, you'd have to know to create a dictionary with Lazy<T> objects inside, so I think it's better as a wrapper class around ConcurrentDictionary<TKey, TValue> .

Answer 2

There's no need for GetOrAdd . A simple check on the path and the key existence is enough:

class FileWorker
{
    private object _sync;
    private IDictionary<string, Task> _destTasks;

    public FileWorker()
    {
        _sync = new object();
        _destTasks = new Dictionary<string, Task>();
    }

    public async Task Copy(IEnumerable<FileInfo> files, string destinationFolder)
    {
        await Task.WhenAll(files.Select(f => Copy(f, destinationFolder)));
    }

    private async Task CreateDestination(string path)
    {
        await Task.Run(() =>
        {
            if (!Directory.Exists(path))
            {
                Directory.CreateDirectory(path);
            }
        });
    }

    private Task Destination(string path)
    {
        lock(_sync)
        {
            if (!_destTasks.ContainsKey(path))
            {
                _destTasks[path] = CreateDestination(path);
            }
        }
        return _destTasks[path];
    }

    private async Task Copy(FileInfo file, string destinationFolder)
    {
        await Destination(destinationFolder).ContinueWith(task => file.CopyTo(Path.Combine(destinationFolder, file.Name), true));
    }
}

class Program
{
    static void Main(string[] args)
    {
        var file1 = new FileInfo("file1.tmp");
        using(var writer = file1.CreateText())
        {
            writer.WriteLine("file 1");
        }
        var file2 = new FileInfo("file2.tmp");
        using(var writer = file2.CreateText())
        {
            writer.WriteLine("file 2");
        }
        var worker = new FileWorker();
        worker.Copy(new[] { file1, file2 }, @"C:\temp").Wait();
        Console.ReadLine();
    }
}