c# HttpClient with proxy

Question

I execute a lot of request to some the resources with HttpClient . To avoid licks I use it as single instance. Something like that... I want to use proxy, so how I can use different proxies for each request?

Thanks!

public class Program
{
    private static HttpClient Client = new HttpClient();
    public static void Main(string[] args)
    {
        Console.WriteLine("Starting connections");
        for(int i = 0; i<10; i++)
        {
            var result = Client.GetAsync("http://aspnetmonsters.com").Result;
            Console.WriteLine(result.StatusCode);
        }
        Console.WriteLine("Connections done");
        Console.ReadLine();
    }

}

Answer 1

You will need to implement IWebProxy.

here is a very row sample.

First implement IWebProxy

public class MyProxy : IWebProxy {
public MyProxy() {  credentials = new NetworkCredential( user, password ); }
private NetworkCredential credentials;
public ICredentials Credentials
{
    get = > credentials;
    set = > throw new NotImplementedException();
}
private Uri proxyUri;
public Uri GetProxy( Uri destination )
{
    return proxyUri; // your proxy Uri
}
public bool IsBypassed( Uri host )
{
    return false;
}
private const string user = "yourusername";
private const string password = "password";}

Then provide it to handler in HttpClient

public class MyHttpClient {
internal static HttpResult httpMethod( ... )
{
    var _client = client();
    try
    {
        var message = new HttpRequestMessage( method, url );
        message.Content = new StringContent( content, Encoding.UTF8, "application/json" );
        var result = _client.SendAsync( message ).Result;// handle result
    }
    catch( Exception e ){}
}
private static HttpClient client()
{
    var httpClientHandler = new HttpClientHandler() { Proxy = new MyProxy() };
    var httpClient = new MyClient( new Uri( "baseurl" ), httpClientHandler );
    return httpClient;

Answer 2

Ah, I misread the question.
It's not about how to use a random IWebProxy with HttpClientHandler, but how to work around being unable to reset the proxy property of the same HttpClientHandler once the first request has started.

The problem is that you can't reset the proxy of the HttpClientHandler...

System.InvalidOperationException: 'This instance has already started one or more requests.
Properties can only be modified before sending the first request.'

But it's still rather easy.

• Proxy property of HttpClientHandler takes an object that implements IWebProxy.
• IWebProxy interface has a method GetProxy that return the Uri of the proxy.
• So you can make your own class that implements this interface and control how it returns the Uri of the proxy with GetProxy.
• You can make it wrap another IWebProxy, and in GetProxy it would return GetProxy of the inner IWebProxy.
• This way, you won't have to change the Proxy property of the HttpClientHandler, you can just change the inner IWebProxy.

Implementation:

public class WebProxyService
  : System.Net.IWebProxy
{
    protected System.Net.IWebProxy m_proxy;


    public System.Net.IWebProxy Proxy
    {
        get { return this.m_proxy ??= System.Net.WebRequest.DefaultWebProxy; }
        set { this.m_proxy = value; }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    { } // Constructor 

    public WebProxyService(System.Net.IWebProxy proxy)
    {
        this.Proxy = proxy;
    } // Constructor 


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return this.Proxy.GetProxy(destination);
    }

    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }


}

And then usage goes like this:

public class AlternatingProxy 
{


    public static async System.Threading.Tasks.Task Test()
    {
        string url = "http://aspnetmonsters.com";
       
        System.Net.WebProxy[] proxies = new[] {
            null,
            new System.Net.WebProxy("104.238.172.20", 8080),
            new System.Net.WebProxy("104.238.167.193", 8080),
            new System.Net.WebProxy("136.244.102.38", 8080),
            new System.Net.WebProxy("95.179.202.40", 8080)
        };

        System.Random rnd = new System.Random();
        WebProxyService proxyService = new WebProxyService();
        
        using (System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient(
          new System.Net.Http.HttpClientHandler { UseProxy = true, Proxy = proxyService }
          ))
        {
            // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent
            hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (iPad; CPU OS 12_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148");
            hc.DefaultRequestHeaders.Add("Accept-Language", "fr-FR, fr;q=0.9, en;q=0.8, it;q=0.7, *;q=0.5");
            hc.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com");
            hc.DefaultRequestHeaders.ConnectionClose = true; 

            for (int i = 0; i < 10; ++i)
            {
                proxyService.Proxy = proxies[rnd.Next(proxies.Length)];
                string response = await hc.GetStringAsync(url);
            }
        }

    } // End Task Test  


} // End Class TestMe 

---

Edit:

If you want to use a singleton, maybe this is an idea:

public class WebProxyService : System.Net.IWebProxy {

    protected System.Net.IWebProxy m_proxy;


    public System.Net.IWebProxy Proxy
    {
        get { return this.m_proxy ??= System.Net.WebRequest.DefaultWebProxy; }
        set { this.m_proxy = value; }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    { } // Constructor 


    public WebProxyService(System.Net.IWebProxy proxy)
    {
        this.Proxy = proxy;
    } // Constructor 


    protected System.Func<System.Net.WebProxy>[] proxies = new System.Func<System.Net.WebProxy>[] {
            delegate(){ return new System.Net.WebProxy("104.238.172.20", 8080); },
            delegate (){ return new System.Net.WebProxy("104.238.167.193", 8080);},
            delegate(){ return new System.Net.WebProxy("136.244.102.38", 8080);},
            delegate(){ return new System.Net.WebProxy("95.179.202.40", 8080);}
        };


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return proxies[RandomGen2.Next(proxies.Length)]().GetProxy(destination);
    }


    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }



    private static class RandomGen2
    {
        private static System.Random _global = new System.Random();

        [System.ThreadStatic]
        private static System.Random _local;

        public static int Next(int maxValue)
        {
            System.Random inst = _local;
            if (inst == null)
            {
                int seed;
                lock (_global) seed = _global.Next();
                _local = inst = new System.Random(seed);
            }
            return inst.Next(maxValue);
        }
    }


} // End Class WebProxyService 

---

Edit 2:

It's still not thread-safe if you change the proxy.
Therefore, using a fixed immutable proxy-list and blocking the set-property.
This way, it should be thread-safe.

public class WebProxyService
      : System.Net.IWebProxy
{


    protected System.Net.IWebProxy[] m_proxyList;

    public System.Net.IWebProxy Proxy
    {
        get
        {
            // https://devblogs.microsoft.com/pfxteam/getting-random-numbers-in-a-thread-safe-way/
            if (this.m_proxyList != null)
                return this.m_proxyList[ThreadSafeRandom.Next(this.m_proxyList.Length)];
            
            return System.Net.WebRequest.DefaultWebProxy;
        }
        set
        {
            throw new System.InvalidOperationException("It is not thread-safe to change the proxy-list.");
        }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    {
    } // Constructor 


    public WebProxyService(System.Net.IWebProxy[] proxyList)
    {
        this.m_proxyList = proxyList;
    } // Constructor 


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return this.Proxy.GetProxy(destination);
    }


    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }


} // End Class WebProxyService 

---

Old answer: ----

Using a proxy with HttpClient in ASP.NET-Core is actually quite simple.
All you need to do is set the handler in the HttpClient-constructor.
Then set the proxy-property of the handler for each request.
Like this:

 public class Program { public static async System.Threading.Tasks.Task Main(string[] args) { string url = "http://aspnetmonsters.com"; System.Net.WebProxy[] proxies = new[] { null, new System.Net.WebProxy("104.238.172.20", 8080), new System.Net.WebProxy("104.238.167.193", 8080), new System.Net.WebProxy("136.244.102.38", 8080), new System.Net.WebProxy("95.179.202.40", 8080) }; System.Random rnd = new System.Random(); using (System.Net.Http.HttpClientHandler handler = new System.Net.Http.HttpClientHandler() { Proxy = new System.Net.WebProxy("http://127.0.0.1:8888"), UseProxy = true, }) { using (System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient(handler)) { System.Console.WriteLine("Starting connections"); for (int i = 0; i < 10; i++) { handler.Proxy = proxies[rnd.Next(proxies.Length)]; await hc.GetAsync(url); // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (iPad; CPU OS 12_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148"); hc.DefaultRequestHeaders.Add("Accept-Language", "fr-FR, fr;q=0.9, en;q=0.8, it;q=0.7, *;q=0.5"); hc.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com"); using (System.Net.Http.HttpResponseMessage response = await hc.GetAsync(url)) { // using (var fs = new System.IO.MemoryStream()) // { await response.Content.CopyToAsync(fs); } byte[] ba = await response.Content.ReadAsByteArrayAsync(); } // End Using response } // Next i System.Console.WriteLine("Ending connections"); } // End Using hc } // End Using handler System.Console.WriteLine("--- Press any key to continue --- "); System.Console.ReadKey(); } // End Task Main } // End Class Program

Answer 3

So basically to be able to change proxies you will need a reference on the HttpClientHandler .
A simple example can be found here: C# use proxy with HttpClient request
and another one here: Simple C# .NET 4.5 HTTPClient Request Using Basic Auth and Proxy

I would suggest to keep the HttpClientHandler on a private field and use the reference to change the proxy everytime you need it.
Keep in mind though that if you need to use different proxies simultaneously you would need to have multiple instances of the HttpClientHandler class.

If you need me to make a sample code for that. Ping me.

Thank you.