Using ThreadPool.QueueUserWorkItem - thread unexpectedly exits

Question

I have the following method:

    public void PutFile(string ID, Stream content)
    {
        try
        {
            ThreadPool.QueueUserWorkItem(o => putFileWorker(ID, content));
        }

        catch (Exception ex)
        {
            OnPutFileError(this, new ExceptionEventArgs { Exception = ex });
        }
    }

The putFileWorker method looks like this:

    private void putFileWorker(string ID, Stream content)
    {
        //Get bucket name:
        var bucketName = getBucketName(ID)
            .ToLower();

        //get file key
        var fileKey = getFileKey(ID);

        try
        {
            //if the bucket doesn't exist, create it
            if (!Amazon.S3.Util.AmazonS3Util.DoesS3BucketExist(bucketName, s3client))
                s3client.PutBucket(new PutBucketRequest { BucketName = bucketName, BucketRegion = S3Region.EU });

            PutObjectRequest request = new PutObjectRequest();
            request.WithBucketName(bucketName)
                .WithKey(fileKey)
                .WithInputStream(content);

            S3Response response = s3client.PutObject(request);
            var xx = response.Headers;

            OnPutFileCompleted(this, new ValueEventArgs { Value = ID });
        }

        catch (Exception e)
        {
            OnPutFileError(this, new ExceptionEventArgs { Exception = e });
        }
    }

I've created a little console app to test this. I wire up event handlers for the OnPutFileError and OnPutFileCompleted events.

If I call my PutFile method, and step into this, it gets to the "//if the bucket doesn't exist, create it" line, then exits. No exception, no errors, nothing. It doesn't complete (i've set breakpoints on my event handlers too) - it just exits.

If I run the same method without the ThreadPool.QueueUserWorkItem then it runs fine...

Am I missing something?

Answer 1

ThreadPool threads are background threads (see the link). They will not keep your application running if the main thread exits.

Typically, in WinForms apps, this is not a problem, because the main UI thread calls Application.Run and starts processing events. For your console app, if your Main method doesn't wait for the work item to complete somehow, the main thread will queue the work item and then exit.

You could create a background thread yourself and set its IsBackground property to false. Or you could create a thread and call Thread.Join to wait for it to finish.

-- EDIT --

As suggested in the comments below, you could also use a ManualResetEvent, or even a custom synchronization class as suggested by Linik. The goal is to block the main thread until the the background threads have completed.

To use a ManualResetEvent, create it in your main thread and pass it in as an argument. (I'll assign it to a static variable here just for brevity.)

ManualResetEvent s_WaitEvent;

ManualResetEvent s_WaitEvent = new ManualResetEvent(false); // non-signaled 
// queue work item here
s_WaitEvent.WaitOne();

At the end of your worker thread, signal the event:

s_WaitEvent.Set();

Link's CountDownLatch is nice if you have many threads that must process before you can exit. You can also use separate ManualResetEvents for each thread and wait for them all to complete using WaitHandle.WaitAll(WaitHandle[]) . (ManualResetEvent inherits from WaitHandle.)

Answer 2

Put a Console.ReadLine() in your Main thread to block it while you test your worker thread. This will keep main from exiting. Just hit enter when you're done.

Answer 3

Use a CountDownLatch to force the main to wait for all of the threads that you have queued up:

public class CountDownLatch 
{
    private int m_remain;
    private EventWaitHandle m_event;

    public CountDownLatch (int count)
    {
        if (count < 0)
            throw new ArgumentOutOfRangeException();
        m_remain = count;
        m_event = new ManualResetEvent(false);
        if (m_remain == 0)
        {
            m_event.Set();
        }
    }

    public void Signal()
    {
        // The last thread to signal also sets the event.
        if (Interlocked.Decrement(ref m_remain) == 0)
            m_event.Set();
    }

    public void Wait()
    {
        m_event.WaitOne();
    }
}

In Main:

static void Main(string[] args)
{
    CountDownLatch latch = new CountDownLatch(numFiles);
    // 
    // ...
    // 
    putFileWorker("blah", streamContent);
    // 
    // ...
    // 

    // waits for all of the threads to signal
    latch.Wait();
}

In the worker method:

private void putFileWorker(string ID, Stream content)
{
    try
    {
        //Get bucket name:
        var bucketName = getBucketName(ID)
            .ToLower();

        //get file key
        var fileKey = getFileKey(ID);

        try
        {
            //if the bucket doesn't exist, create it
            if (!Amazon.S3.Util.AmazonS3Util.DoesS3BucketExist(bucketName, s3client))
                s3client.PutBucket(new PutBucketRequest { BucketName = bucketName, BucketRegion = S3Region.EU });

            //
            // ... 
            // 
        }

        catch (Exception e)
        {
            OnPutFileError(this, new ExceptionEventArgs { Exception = e });
        }
    }
    finally
    {
        latch.Signal();
    }
}