Issue in asynchronous API calls from SQLCLR

Question

In a nutshell, I need to notify a Web API service from SQL Server asynchronously as and when there are changes in a particular table.

To achieve the above, I have created a SQLCLR stored procedure which contains the asynchronous API call to notify the service. The SQLCLR stored procedure is called via a trigger as and when there is an insert into a table called Table1 . The main challenge here is API has to read data from the same table ( Table1 ).

If I use HttpWebRequest.GetResponse() which is the synchronous version, the entire operation is getting locked out due to the implicit lock of the insert trigger. To avoid this, I have used HttpWebRequest.GetResponseAsync() method which calls the API and doesn't wait for the response. So it fires the API request and the program control moves on so the trigger transaction doesn't hold any lock(s) on table1 and the API was able to read data from table1 .

Now I have to implement an error notification mechanism as and when there are failures (like unable to connect to remote server) and I need to send an email to the admin team. I have wrote the mail composition logic inside the catch() block. If I proceed with the above HttpWebRequest.GetResponseAsync().Result method, the entire operation becomes synchronous and it locks the entire operation.

If I use the BeginGetResponse() and EndGetResponse() method implementation suggested in Microsoft documents and run the SQLCLR stored procedure, SQL Server hangs without any information, why? What am I doing wrong here? Why does the RespCallback() method not get executed?

Sharing the SQLCLR code snippets below.

public class RequestState
{
    // This class stores the State of the request.
    // const int BUFFER_SIZE = 1024;
    // public StringBuilder requestData;
    // public byte[] BufferRead;
    public HttpWebRequest request;
    public HttpWebResponse response;
    // public Stream streamResponse;

    public RequestState()
    {
        // BufferRead = new byte[BUFFER_SIZE];
        // requestData = new StringBuilder("");
        request = null;
        // streamResponse = null;
    }
}

public partial class StoredProcedures
{
    private static SqlString _mailServer = null;
    private static SqlString _port = null;
    private static SqlString _fromAddress = null;
    private static SqlString _toAddress = null;
    private static SqlString _mailAcctUserName = null;
    private static SqlString _decryptedPassword = null;
    private static SqlString _subject = null;

    private static string _mailContent = null;
    private static int _portNo = 0;

    public static ManualResetEvent allDone = new ManualResetEvent(false);
    const int DefaultTimeout = 20000; // 50 seconds timeout

#region TimeOutCallBack
/// <summary>
/// Abort the request if the timer fires.
/// </summary>
/// <param name="state">request state</param>
/// <param name="timedOut">timeout status</param>
private static void TimeoutCallback(object state, bool timedOut)
{
if (timedOut)
{
HttpWebRequest request = state as HttpWebRequest;
if (request != null)
{
request.Abort();
SendNotifyErrorEmail(null, "The request got timedOut!,please check the API");
}
}
}
#endregion

#region APINotification
[SqlProcedure]
public static void Notify(SqlString weburl, SqlString username, SqlString password, SqlString connectionLimit, SqlString mailServer, SqlString port, SqlString fromAddress
, SqlString toAddress, SqlString mailAcctUserName, SqlString mailAcctPassword, SqlString subject)
{
_mailServer = mailServer;
_port = port;
_fromAddress = fromAddress;
_toAddress = toAddress;
_mailAcctUserName = mailAcctUserName;
_decryptedPassword = mailAcctPassword;
_subject = subject;

if (!(weburl.IsNull && username.IsNull && password.IsNull && connectionLimit.IsNull))
{
var url = Convert.ToString(weburl);
var uname = Convert.ToString(username);
var pass = Convert.ToString(password);
var connLimit = Convert.ToString(connectionLimit);
int conLimit = Convert.ToInt32(connLimit);
try
{
if (!(string.IsNullOrEmpty(url) && string.IsNullOrEmpty(uname) && string.IsNullOrEmpty(pass) && conLimit > 0))
{
SqlContext.Pipe.Send("Entered inside the notify method");

HttpWebRequest httpWebRequest = WebRequest.Create(url) as HttpWebRequest;
string encoded = Convert.ToBase64String(Encoding.GetEncoding("ISO-8859-1").GetBytes(uname + ":" + pass));
httpWebRequest.Headers.Add("Authorization", "Basic " + encoded);
httpWebRequest.Method = "POST";
httpWebRequest.ContentLength = 0;
httpWebRequest.ServicePoint.ConnectionLimit = conLimit;

// Create an instance of the RequestState and assign the previous myHttpWebRequest
// object to its request field. 
RequestState requestState = new RequestState();
requestState.request = httpWebRequest;

SqlContext.Pipe.Send("before sending the notification");
//Start the asynchronous request.
IAsyncResult result =
(IAsyncResult)httpWebRequest.BeginGetResponse(new AsyncCallback(RespCallback), requestState);
SqlContext.Pipe.Send("after BeginGetResponse");

// this line implements the timeout, if there is a timeout, the callback fires and the request becomes aborted
ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), requestState, DefaultTimeout, true);
//SqlContext.Pipe.Send("after RegisterWaitForSingleObject");

// The response came in the allowed time. The work processing will happen in the 
// callback function.
allDone.WaitOne();
//SqlContext.Pipe.Send("after allDone.WaitOne();");

// Release the HttpWebResponse resource.
requestState.response.Close();
SqlContext.Pipe.Send("after requestState.response.Close()");
}
}
catch (Exception exception)
{
SqlContext.Pipe.Send(" Main Exception");
SqlContext.Pipe.Send(exception.Message.ToString());
//TODO: log the details in a error table
SendNotifyErrorEmail(exception, null);
}
}
}
#endregion

#region ResposnseCallBack
/// <summary>
/// asynchronous Httpresponse callback
/// </summary>
/// <param name="asynchronousResult"></param>
private static void RespCallback(IAsyncResult asynchronousResult)
{
try
{
SqlContext.Pipe.Send("Entering the respcallback");
// State of request is asynchronous.
RequestState httpRequestState = (RequestState)asynchronousResult.AsyncState;
HttpWebRequest currentHttpWebRequest = httpRequestState.request;
httpRequestState.response = (HttpWebResponse)currentHttpWebRequest.EndGetResponse(asynchronousResult);
SqlContext.Pipe.Send("exiting the respcallBack");
}
catch (Exception ex)
{
SqlContext.Pipe.Send("exception in the respcallBack");
SendNotifyErrorEmail(ex, null);
}
allDone.Set();
}
#endregion
}

One alternative approach for above is using SQL Server Service Broker which has the queuing mechanism which will help us to implement asynchronous triggers. But do we have any solution for the above situation? Am I doing anything wrong in an approach perspective? Please guide me.

Answer 1

There are a couple of things that stand out as possible issues:

1. Wouldn't the call to allDone.WaitOne(); block until a response is received anyway, negating the need / use of all this async stuff?
2. Even if this does work, are you testing this in a single session? You have several static member (class-level) variables, such as public static ManualResetEvent allDone , that are shared across all sessions. SQLCLR uses a shared App Domain (App Domains are per Database / per Assembly owner). Hence multiple sessions will over-write each other's values of these shared static variables. That is very dangerous (hence why non -readonly static variables are only allowed in UNSAFE Assemblies). This model only works if you can guarantee a single caller at any given moment.

Beyond any SQLCLR technicalities, I am not sure that this is a good model even if you do manage to get past this particular issue.

A better, safer model would be to:

1. create a queue table to log these changes. you typically only need the key columns and a timestamp ( DATETIME or DATETIME2 , not the TIMESTAMP datatype).
2. have the trigger log the current time and rows modified into the queue table

3. create a stored procedure that takes items from the queue, starting with the oldest records, processes them (which can definitely be calling your SQLCLR stored procedure to do the web server call, but no need for it to be async, so remove that stuff and set the Assembly back to EXTERNAL_ACCESS since you don't need/want UNSAFE ).

   Do this in a transaction so that the records are not fully removed from the queue table if the "processing" fails. Sometimes using the OUTPUT clause with DELETE to reserve rows that you are working on into a local temp table helps.

   Process multiple records at a time, even if calling the SQLCLR stored procedure needs to be done on a per-row basis.

4. create a SQL Server agent job to execute the stored procedure every minute (or less depending on need)

Minor issues:

1. the pattern of copying input parameters into static variables (eg _mailServer = mailServer; ) is pointless at best, and error-prone regardless due to not being thread-safe. remember, static variables are shared across all sessions, so any concurrent sessions will be overwriting the previous values, hence ensure race conditions. Please remove all of the variables with names starting with an underscore.
2. the pattern of using Convert.To... is also unnecessary and a slight hit to performance. All Sql* types have a Value property that returns the expected .NET type. Hence, you only need: string url = weburl.Value;
3. there is no need to use incorrect datatypes that require conversion later on. Meaning, rather than using SqlString for connectionLimit , instead use SqlInt32 and then you can simply do int connLimit = connectionLimit.Value;
4. You probably don't need to do the security manually (ie httpWebRequest.Headers.Add("Authorization", "Basic " + encoded); ). I am pretty sure you can just create a new NetworkCredential using uname and pass and assign that to the request.

Answer 2

Hi there (and Happy New Year) :)!

A couple of things:

• If you can, stay away from triggers! They can cause a lot of unexpected side effects, especially if you use them for business logic. What I mean with that, is that they are good for auditing purposes, but apart from that I try and avoid them like the plague.

• So, if you do not use triggers, how do you know when something is inserted? Well, I hope that your inserts happen through stored procedures, and are not direct inserts in the table(s). If you use stored procedures you can have a procedure which is doing your logic, and that procedure is called from the procedure which does the insert.

Back to your question. I don't really have an answer to the specifics, but I would stay away from using SQLCLR in this case (sidenote: I am a BIG proponent of SQLCLR, and I have implemented a lot of SQLCLR processes which are doing something similar to what you do, so I don't say this because I do not like SQLCLR).

In your case, I would look at either use Change Notifications , or as you mention in your post - Service Broker. Be aware that with SSB you can run into performance issues (latches, locks, etc.) if your system is highly volatile (+2,000 tx/sec). At least that is what we experienced.