Execute job on server and redirect user to different page

Question

I have a process that users run from my website. Sometimes they select a small amount of data and other times it's a large amount. What I want to do is if its a large amount when the user submits the job I want to start the process of creating the data set on the web server and when the job finishes I have a function to check for errors. At the same time that this process starts I want to redirect the user to another page that says they will receive an email when the job is complete.
Currently I'm doing it in a synchronously which only works on the small jobs. The big jobs time out after 5 minutes on firefox 30. Here is my process code

try
{
  Process p = new Process();
  p.StartInfo.Arguments = " -config 'C:\\temp\\Web.cfg'" ;
  p.StartInfo.CreateNoWindow = true;
  p.StartInfo.FileName = "sas.exe";
  p.StartInfo.RedirectStandardOutput = true;
  p.StartInfo.UseShellExecute = false;
  p.Start();
  p.WaitForExit();
  p.Close();
}
catch (Exception ex)
{
  Console.WriteLine(ex.Message);
}

errDesc = clsUtil.CheckErrors("test.log", "ERROR");

I'm very new to this type of web programing so I apologize if I miss something obvious.

Thank you

EDIT:

Here is the code i'm currently using and it seems to work. Do you see any issues arising from this code? I tested it with 4 browsers running big jobs and they are redirected after submit and I received the emails I expected that the jobs are complete. Do I need to dispose of the thread?

if ((nCols > nColsMax) || (ckbEmail.Checked == true))
{
 Thread tr = new Thread(() => MakeDataThread(cartID));
 tr.Start();
 Response.Redirect("../EmailID.aspx?Email=" + Session["User"].ToString());
}

public static void MakeDataThread(int iCartID) 
{
 clsUtil uUtil = new clsUtil();
 uUtil.RunAJobThread(iCartID);
}

Within uUtil.RunAJobThread I take the cartId and go to the oracle database and select the data I think create a document then I send the user an email.

Answer 1

As long as the jobs aren't stepping on top of each other for a shared resource (like a file etc...) You have to do this with care and test it but yes what you need to do is turn your method into an async or make your method thread out the call that invokes this process code.

You would probably want to make a db table to support this, upon the web request generate a row in this new table (eg Task). Generate a TaskId, use this id to update/report status.

The taskId would be sent back to the web client via a redirect and that TaskId could be used to query status. Or you could use session if this is a single server or you don't want to put the back end to it.

To summarize:

• Receive request
• Generate a TaskId (DB or some static factory of your own)
• Send that TaskId to the AsyncTask so it can use it to report progress. (This will execute under a new thread)
• Send a redirect with the TaskId to client ( this will happen almost immediately..)

Now your client can call a page with the taskId and you can reply with status while the AsyncTask is processing/completing

Answer 2

What I've done in the past for long running processes executed from a web page is to:

1) Write the user's data input values to a table.

2) Notify the user that they will be notified when the data is ready. When they confirm this, they are redirected wherever you want them to go.

3) Now, you can perhaps write a console application that performs the long-running process. Run the console application from the server however often you need to.

4) Finally, when the data is processed, notify the user via email.