Why does sending many emails asynchronously block the site? ( using Action.BeginInvoke )

Question

I created a page to send thousands of emails to our clients, almost 8K emails. The sending process is taking hours, but after a while I couldn't access any page (get waiting...) in the site that is hosting the page except for static files (images etc...).

Using: IIS 6 and .Net 4.0

Code:

public static bool Send(MailSettings settings, Action<string, string[], bool> Sent = null)
{
  System.Net.Mail.SmtpClient client; 
...
  foreach(){
    try{ client.Send(message);}catch{...client.Dispose();...}
    Sent.BeginInvoke(stringValue, stringArray, boolValue, null, null);
    if(count++>N){
       count=1;
       System.Threading.Thread.Sleep(1000);
    }
  }
...
}

public void SentComplete(string email, string[] value, bool isSent)
{
   ....//DB logging
}

• Note: Other sites using the same Application pool were fine!

Questions:

• Is there a IIS 6.0 limitation to the number of threads for the same website?

• Any Ideas if my code was causing any performance issues? Am I using Action right?

Answer 1

There are many things wrong with this code:

1. Your try catch block is very weird. You are disposing an object in one iteration but it can be used by others. Try using block instead.
2. There is a maximum time for asp.net request to execute.
3. Don't put thread sleep in asp.net!
4. Yes, there is a maximum thread count in asp.net pool

If you end up blocking for one reason, you can also be blocked by the session (if you have one)... Does it work if you open a different browser?

Answer 2

You're firing off a whole bunch of actions to be executed in the thread pool. There is a max number of threads that the thread pool will create, after that the work items sent to the pool simply get queued up. Since you're flooding the thread pool with so many operations you're preventing the thread pool from ever having an opportunity to get a chance to work on the items added by ASP.NET to handle pages. Since static items don't need to push work to the thread pool, those items can be serviced.

You shouldn't be firing off so many items in parallel. You should be limiting the degree of parallelism to a reasonably small fixed amount. Let those handful of items that you start each process a large number of operations that you have so that the threads in the thread pool have the possibility of working on other things as well.

Answer 3

We regularly send 10000 client emails. I store all the details of the email in a database and then call a web service to send them. This just chuggs through them and affects nothing else. I do put the thread to sleep (in the web service) for 100ms between each call to Send ... if I don't do this firing off so many emails seems to overwhelm our mail server and we get some odd things happening.