SSIS Script Component Blocking

Question

I have a script transform as part of a dataflow which calls a webservice and writes the result to the output buffer for use later on in the pipeline. For some reason it acts as if it is a blocking component as nothing will exit the component until all of the rows have been processed. I was under the impression that the Script Component is synchronous and as such non-blocking. I suspect I must be doing something in my code to cause this but haven't had any success trying to nail it down.

This is the script:

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using SC_fbaeef4508eb4003b6e051eac81d22c3.net.integrationpoint.dps;
using System.Net;
using System.Xml;


[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    private string username;
    private string company;
    private string password;
    private string dtsSearchFlag;
    private string CompanySync;
    private int counter;
    
    public override void PreExecute()
    {
        base.PreExecute();

        //parameters - same for every row
        company = Variables.Company;
        username = Variables.UserName;
        password = Variables.Password;
        dtsSearchFlag = Variables.SearchFlag;
        CompanySync = Variables.CompanySync;
        counter = 0;
    }

    public override void Input0_ProcessInputRow(Input0Buffer Row)
    {
        try
        {
            //input columns
            string companyId = Row.DPSCompanyID ;
            string name = Row.DPSCompanyName ;
            string address = Row.DPSAddress ;
            string city = Row.DPSCity ;
            string countryStateCode = Row.DPSState ;
            string countryCode = Row.DPSCountryCode ;
            string postalCode = Row.DPSPostalCode ;

            //call webservice 
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
            DTSWebService service = new DTSWebService();

            XmlNode result = service.DPSIntegratedSearch(company,
                username,
                password,
                "",                     //loggedInUser
                companyId,
                "",                     //SearchType 
                "",                     //settingsDescription
                name,
                address,
                city,
                countryCode,
                countryStateCode,
                postalCode,
                dtsSearchFlag,
                "",                     //dtsLastValidatedDate 
                "",                     //dtsOverride
                "",                     //dtsOverrideDate
                "",                     //SearchRefNum
                CompanySync
            );

            //get results
            XmlNodeList nodes = result.SelectNodes("//search_status");
            XmlNode node = nodes[0];

            string resultGUID = node["result_guid"].InnerText;
            string dtsSearchDate = node["dts_search_date"].InnerText;
            string dtsMatch = node["dts_match"].InnerText;
            string dtsLastValidatedDate = node["dts_last_validated_date"].InnerText;
            string dtsOverride = node["dts_override"].InnerText;
            string dtsOverrideDate = node["dts_override_date"].InnerText;
            ushort hits = Convert.ToUInt16(node["num_hits"].InnerText);

            //write to output
            Row.ResultGUID = new Guid(resultGUID);
            Row.SearchDate = Convert.ToDateTime(dtsSearchDate);
            Row.Match = dtsMatch;
            Row.LastValidatedDate = Convert.ToDateTime(dtsLastValidatedDate);
            Row.Override = dtsOverride;
            Row.OverrideDate = Convert.ToDateTime(dtsOverrideDate);
            Row.NumHits = hits;

            //counter (debug only)
            counter++;
           
        }
        catch (Exception ex)
        {
            if (ex.InnerException != null)
            {
                OutputErrBuffer.ErrorMessage = ex.Message + " : " + ex.InnerException.InnerException.ToString().Substring(0, 3500) + " ...";
             }
            else if (ex.StackTrace != null)
            {
                OutputErrBuffer.ErrorMessage = ex.Message + " : " + (ex.StackTrace.Length > 3500 ? ex.StackTrace.ToString().Substring(0, 3500) + " ..." : ex.StackTrace.ToString().Substring(0, ex.StackTrace.Length));
            }
            else
            {
                OutputErrBuffer.ErrorMessage = ex.Message;
            }
            ComponentMetaData.FireWarning(1, "CallWebservice", "Exception calling webservice for AddressID:",null, 0);
        }
    }
}

And here is the behavior:

Answer 1

I'm actually not sure it will work with transformations, but:

• Open the properties dialog of your destination component;
• If its "Data access mode" is Table or view - fast load then you will have access to the "Rows per batch" property. Here, you can specify the amount of records that will comprise a single batch.

This value is propagated up the dataflow and affects the data sources, at least. With luck, it will also be applied to your transformation component.

Answer 2

Yes, by default, the Script Component is working in synchronous mode, but it still use buffers to keep some rows/data. You can limit the buffer size with DefaultBufferMaxRow and DefaultBufferSize properties in the data flow task. In your case, set DefaultBufferMaxRow to 1 (the default is 10k)

Note: It might still get a bit more rows to process at once. Just add few more columns to the select clause in your source component and it will work.