Unable to connect to a Web page from SSIS (SQL Server)

Question

Alright! After a lot of research on web, I couldn't find any of the solution working for me. I am trying to connect to a Web page from SSIS and I've tried both http connection manager and a Script Task but none of them seems to be working.

I get the following errors:

Through Script Component: Exception has been thrown by the target of an invocation.

Through http connection manager: The remote server returned an error: (503) Server Unavailable.

Here is one of the VB code I've tried:

Public Sub Main()
        Dim MyWebClient As New System.Net.WebClient()
        Dim proxyObject As New System.Net.WebProxy("BANANA:8080", True)
        MyWebClient.Proxy = proxyObject
        MyWebClient.Proxy.Credentials = CredentialCache.DefaultCredentials

        Dim remoteURi As String
        remoteURi = "http://www.fhfa.gov/DataTools/Downloads/Pages/Conforming-Loan-Limits.aspx"
        Dim filename As String
        filename = "C:\LoanLimits.html"
        MyWebClient.DownloadFile(remoteURi, filename)

    End Sub

And, here is one of the C# codes I've tried:

        public void Main()
    {
        Variables varCollection = null;

        Dts.VariableDispenser.LockForRead("User::RemoteUri");
        Dts.VariableDispenser.LockForRead("User::LocalFolder");
        Dts.VariableDispenser.GetVariables(ref varCollection);
        varCollection.Unlock();

        System.Net.WebClient myWebClient = new System.Net.WebClient();
        string webResource = varCollection["User::RemoteUri"].Value.ToString();
        string fileName = varCollection["User::LocalFolder"].Value.ToString() + webResource.Substring(webResource.LastIndexOf('/') + 1);
        myWebClient.DownloadFile(webResource, fileName);

        Dts.TaskResult = (int)ScriptResults.Success;
    }

I could open the same web page mentioned in the code manually in all my browsers (IE, Firefox, Chrome).

FYI, I am using SQL Server 2014. Please let me know if you need any other information that I can give you to help me.

Answer 1

I cannot reproduce your issue both via Http connection manager and Script Task.

In Http Connection Manager, I simply pasted the uri you provided ( http://www.fhfa.gov/DataTools/Downloads/Pages/Conforming-Loan-Limits.aspx ) and test connection succeeded.

In Script Task, I just use the simplest code:

    public void Main()
    {
        // TODO: Add your code here
        var client = new WebClient();
        client.DownloadFile(new Uri("http://www.fhfa.gov/DataTools/Downloads/Pages/Conforming-Loan-Limits.aspx"), "D:\\test.html");
        Dts.TaskResult = (int)ScriptResults.Success;
    }

And the script task also successfully download the file.

I tested both on SSDT-BI 2014 and SSDT 2016. And I've also tried to deploy and run the package with Script Task on SQL Server 2014, the result was also successful.

So I want to know if the issue happened on your SSIS designer or SQL Server. If it just happened on your SQL Server, maybe you should check the network setting of your SQL Server. If your SQL Server is OK, I think you can change a machine to verify if the issue happens again.