Making worker threads?

Question

Is there a way to make "worker threads" Basically I tried creating threads every time I needed them and this resulted in being slower than with 1 thread becase creating a new thread all the time is expensive. Would there be a way to create worker threads when the application first starts, then give them work when necessary?

Thanks

Answer 1

Yes, you can create the threads up front and have them wait for signals to go and start their work.

These signals can be message queues or semaphores or any other sort of inter-thread communication method.

As an example, we once put together a system under UNIX which had a master thread to receive work over the net and farm the jobs out to slave threads. The slave threads would actually have to interact with another system by screen scraping (basically emulating a user) to get their desired information.

In other words, work could come in faster than slave threads could do it.

So we started up about fifty odd slaves and just created a condition variable for them (this was using pthreads). Each slave that wasn't doing active work simply waited on the condition variable.

When a job came in to the master thread, it loaded the relevant data into known memory and "kicked" the condition variable, waking up one of the slaves which would then grab the work and start processing it (after notifying the master that it could continue).

So there was no overhead in having to create threads on the fly - all were created on application start-up and simply waited for work to be handed over.

Of course, there's a potential downside to this sort of static sizing in that you may get into trouble if you actually need more threads. We solved it by simply monitoring the maximum number of threads and ensuring the process was restarted the following day with more if we were consistently running out.

---

If you want to have a more dynamic solution, you need to look into a concept called thread pooling. This is basically what I've described above but it generally lets you set a minimum and maximum number of threads along with a maximum time an inactive thread will be allowed to survive without work (unless you're already at the minimum).

Its implementation could be something as simple as:

master:
    minslaves = 7
    maxslaves = 20
    maxtime = 600
    numslaves = 0
    freeslaves = 0

    while running:
        while numslaves < minslaves:
            increment numslaves and freeslaves
            start a slave
        endwhile
        wait for work
        if freeslaves = 0 and numslaves < maxslaves:
            start a slave
        endif
        queue work to slavequeue
    endwhile
    while numslaves > 0:
        wait
    endwhile
    exit

and:

slave:
    while running:
        get work from slavequeue with timeout
        if timed out:
            if time since last work > maxtime and numslaves > minslaves:
                break out of while loop
            endif
            restart while loop
        endif
        decrement freeslaves
        do the work
        increment freeslaves
    endwhile
    decrement numslaves and freeslaves
    exit

(with proper semaphore protection of all shared variables of course, numslaves , freeslaves , slavequeue and the others if you decide to change them after threads have been started).

And, if your environment already has thread pooling, I suggest you use it rather than trying to convert my pseudo-code above. My code is from memory and is meant to illustrate a point - it may or may not be bug-free :-)

Answer 2

This is usually done by creating message queues. A thread would wait on some object, signaling of which will indicate there's new work to complete.

Answer 3

On win32 there is a very good threadpool - esp vista and windows 7 onwards. I've always found it useful to think in terms of tasks that need to be performed concurrently than thinking of individual threads.

Also, the parallel patterns library (ConcRT I think it is called) that comes with Visual C++ 2010 looks very powerful (and has a simple interface). I haven't used it yet myself though. The MSDN link is - http://msdn.microsoft.com/en-us/library/dd492418.aspx