Multi-threaded Queue in C#

Question

I've been tasked with working on a download queuing system but I'm a bit confused about where to start.

Essentially what we need to do is to have something like a download manager (but not as fully blown). We have about 20-100 files to download, we give the user a UI (with a listview) to allow them to pause, stop, or move the priorty of jobs around.

What I'm confused about is the data-structure to use, a Priority Queue seems like the way to go from my research, but I'm confused about how to make it work. Do I have a background thread that peeks into the Queue and picks up the next task and carries it forward? I need to provide progress too as the files are being downloaded - they are quite large, sometimes 120Mb (but its local, so no more than 10mins).

Sometimes they need to pause a job and shove a job higher up in the queue as its deemed urgent.

Its not a download manager, so no throttling etc issues. How do people write things like this?

I was thinking of having an interface like IDownloadTask which describes the task to carry out, have a few properties and an event to expose its Progress (which gets wired up when the tasks runs).

Then put that IDownloadTask into the queue with a priority. A background worker picks it up (the PriorityQUeue will need to be synchronised I guess) and then executes the .Execute() method in the interface implementation on a seperate thread.

Does this sound reasonable? Are there any concrete examples anyone can show me somewhere?

EDIT

Wow thanks for the reply and the vote of confidence, I should mention that I'm using .NET 2.0 (we can't move higher because of Windows compatibility requirements for Windows 9x).

Answer 1

As for tracking progress, your thread can report progress using events, as well as completion. Here is an example with a completion event, but the same concept would work for a Status update event. You'd just change the class that holds the data so that it can pass info about progress.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace ThreadWithDataReturnExample
{
    public partial class Form1 : Form
    {
        private Thread thread1 = null;

        public Form1()
        {
            InitializeComponent();

            thread1 = new Thread(new ThreadStart(this.threadEntryPoint));
            Thread1Completed += new AsyncCompletedEventHandler(thread1_Thread1Completed);
        }

        private void startButton_Click(object sender, EventArgs e)
        {
            thread1.Start();
            //Alternatively, you could pass some object
            //in such as Start(someObject);
            //With apprioriate locking, or protocol where
            //no other threads access the object until
            //an event signals when the thread is complete,
            //any other class with a reference to the object 
            //would be able to access that data.
            //But instead, I'm going to use AsyncCompletedEventArgs 
            //in an event that signals completion
        }

        void thread1_Thread1Completed(object sender, AsyncCompletedEventArgs e)
        {
            if (this.InvokeRequired)
            {//marshal the call if we are not on the GUI thread                
                BeginInvoke(new AsyncCompletedEventHandler(thread1_Thread1Completed),
                  new object[] { sender, e });
            }
            else
            {
                //display error if error occurred
                //if no error occurred, process data
                if (e.Error == null)
                {//then success

                    MessageBox.Show("Worker thread completed successfully");
                    DataYouWantToReturn someData = e.UserState as DataYouWantToReturn;
                    MessageBox.Show("Your data my lord: " + someData.someProperty);

                }
                else//error
                {
                    MessageBox.Show("The following error occurred:" + Environment.NewLine + e.Error.ToString());
                }
            }
        }

        #region I would actually move all of this into it's own class
            private void threadEntryPoint()
            {
                //do a bunch of stuff

                //when you are done:
                //initialize object with data that you want to return
                DataYouWantToReturn dataYouWantToReturn = new DataYouWantToReturn();
                dataYouWantToReturn.someProperty = "more data";

                //signal completion by firing an event
                OnThread1Completed(new AsyncCompletedEventArgs(null, false, dataYouWantToReturn));
            }

            /// <summary>
            /// Occurs when processing has finished or an error occurred.
            /// </summary>
            public event AsyncCompletedEventHandler Thread1Completed;
            protected virtual void OnThread1Completed(AsyncCompletedEventArgs e)
            {
                //copy locally
                AsyncCompletedEventHandler handler = Thread1Completed;
                if (handler != null)
                {
                    handler(this, e);
                }
            }
        #endregion

    }
}

Answer 2

Here are two C# projects that you can probably use to get you started.

• MyDownloader: A Multi-thread C# Segmented Download Manager
• Multi-threaded file download manager

Answer 3

Here are is a mini implementation you can start out with:

C# Threading issue with AutoResetEvent

You will probably want to have more that 1 processing thread, and you will probably need to add some comms back to the piece of data being processed so you can pause etc ...