ThreadPool Thread needs to wait

Question

I have a application which loads of messages need to be handled (eg: 2000 per second). And the business requirement requires me not to handle the message immediately , but to wait for 2 seconds to handle every single message. what I'm doing now is to spin off a thread from thread pool via Task.Factory.StartNew for every message, and do the "waiting 2 seconds" job inside the thread from the pool. The problem is when the message load is really high, I always get OutOfMemory exception , although the memory consumption is not actually that high according to Task Manager from Windows OS. However if I don't wait inside the thread, then everything is ok.

My guess is that when the message load is high, the thread pool threads are not enough to handle all the messages. thus more and more messages are queued to wait to be handled, when the queue size gets really big, it results in a OOM exception . Have already tried ThreadPool.SetMaxThreads and ThreadPool.SetMinThreads to very high number, but still doesn't work. Any advices?

The code looks like this :

 ThreadPool.SetMaxThreads(32768, 32768);
 ThreadPool.SetMinThreads(2500, 2500);

public void HanldeMessage(string message)
{
     Task.Factory.StartNew(() => DoWork(message))
}

public void DoWork(string message)
{
   Thread.Sleep(2000);
   // do some work to message
}

Answer 1

Try this. You'll need .NET 4.5.

private static void HandleMessage(string message)
{
    DoWork(message);
}

private static async Task DoWork(string message)
{
    await Task.Delay(2000); // instead of thread.Sleep

    // do some work...
    Console.WriteLine(message);
}

Instead of blocking on thread.sleep, this should effectively return the thread to the pool for 2 seconds, and return execution afterward.

I recommend you do not block threads, especially when dealing with a scenario where thousands may be needed. I'd also recommend you not set ThreadPool.MaxThreads/MinThreads. You are putting it through a lot of abuse here.

Each thread takes up 1MB of stack space. I'd wager the reason you see the OOM exceptions is because you are creating too many threads, not because of queued requests to make threads. Your situation above is awful, because after going through all the work to create a thread, it just sits and rots for 2 seconds. More work is requested, and the thread-pool does its darnedest to help out by creating another. With the amount of requests you expect, its only a matter of time til memory becomes the barrier.