How do I increase the performance of HttpWebResponse?

Question

I am trying to build an application that sends and receives responses from a website.

None of the solutions I've read on Stack Overflow have solved my problem, so I think that my code could use optimization.

I have the following thread:

void DomainThreadNamecheapStart()
{
    while (stop == false)
    {
        foreach (string FromDomainList in DomainList.Lines)
        {
            if (FromDomainList.Length > 1)
            {
                // I removed my api parameters from the string
                string namecheapapi = "https://api.namecheap.com/foo" + FromDomainList + "bar";

                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(namecheapapi);
                request.Proxy = null;
                request.ServicePoint.Expect100Continue = false;

                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                StreamReader sr = new StreamReader(response.GetResponseStream());

                status.Text = FromDomainList + "\n" + sr.ReadToEnd();

                sr.Close();
            }
        }
    }
}

This thread is called when a button is clicked:

private void button2_Click(object sender, EventArgs e)
{
    stop = false;
    Thread DomainThread = new Thread(new ThreadStart(DomainThreadNamecheapStart));
    DomainThread.Start();
}

I only receive around 12 responses in 10 seconds using the above code. When I try to make the same request in JavaScript or using a simple IFrame, it's more than twice as fast. The Browser doesn't use multiple threads for the connection, it waits until one is finished and then starts the new one.

I tried setting request.Proxy = null; , but it had negligible impact.

I have noticed that HTTPS is 2-3 times slower than HTTP . Unfortunately, I have to use HTTPS . Is there anything I can do to make it faster?

Answer 1

My bet would be on the aspect you pointed out - the HTTPS protocol.

The iteration between Client(browser) and Server for pure HTTP protocol is quite straightforward: Ask for the info, get info. If 1.0, close connection; if 1.1, it may stay alive for reuse. (Check image 1 for details.)

But when you do a HTTPS request, the initial protocol overhead is considerable (image 2); but, once the initial negotiation is done, some symmetric encryption takes place, and no further certificate negotiation is necessary, thus speeding up data transfer.

What I think the problem is, if you destroy the HTTPWebRequest object and creates a new one, the full HTTPS exchange takes place once again, slowing your iteration. (HTTPS + HTTP 1.1 Keepalive should do fine, though.)

So, suggestions: Switch to HTTP only, or reuse the connection objects.

And i hope it works for you. =)

(1) HTTP protocol handshake and response

(2) Initial HTTPS protocol handshake

Answer 2

Try to modify the System.Net.ServicePointManager.DefaultConnectionLimit value (the default value is 2).

Other reference ( Performance Issues Part).

Answer 3

try these, it helped me to improve the performance,

        ServicePointManager.Expect100Continue = false;
        ServicePointManager.DefaultConnectionLimit = 200;
        ServicePointManager.MaxServicePointIdleTime = 2000;
        ServicePointManager.SetTcpKeepAlive(false, 0, 0);