Reading a website using C# Agility Pack from a proxied server application hosted in IIS 7

Question

I am creating an ASP.NET MVC 4 website that retrieves offers from other sites and aggregates their content in my website. (Basically, it's just food offers for me and some work friends to use when we go out to lunch.)

I am using Agility Pack to read the websites as follows:

HtmlWeb webget = new HtmlWeb();
WebProxy wp = new WebProxy(new Uri("http://my-proxy.proxy.proxy.proxy:<port>"), true);
wp.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

var doc = webget.Load(url, "GET", wp,
    System.Net.CredentialCache.DefaultNetworkCredentials);

Running in Visual Studio it works. But when i run it from IIS , I get the following error:

ERROR: The requested URL could not be retrieved. The following error was encountered while trying to retrieve the URL: access denied . Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect. Your administrator of the cache is generated Tue, 25 Nov 2014 12:22:30 GMT by proxy Address (squid).

Note that I have configured my web application with an ASP.NET v4.0 application pool.

Is there anything missing from my code?

Answer 1

The Identity of the Application pool does not have the required permission to access the proxy server.

You can either set the identity of your application pool to a user that would have permissions, or set the identity of a user within the code.

var request = WebRequest.Create("http://foo.bar/file.doc");
request.Credentials = new System.Net.NetworkCredential("username", "password");