[英]Proxy works locally but fails when uploaded to webhost
I have spent a good time now on configuring my proxy. 我现在花了很多时间来配置我的代理。 At the moment I use a service called proxybonanza. 目前我使用的是名为proxybonanza的服务。 They supply me with a proxy which I use to fetch webpages. 他们为我提供了一个用于获取网页的代理。
I'm using HTMLAGILITYPACK 我正在使用HTMLAGILITYPACK
Now if I run my code without a proxy there's no problem locally or when uploaded to webhost server. 现在,如果我在没有代理的情况下运行我的代码,那么在本地或上传到webhost服务器时没有问题。
If I decide to use the proxy, it takes somewhat longer but it stills works locally. 如果我决定使用代理,它需要更长的时间,但它仍然在本地工作。
If I publish my solution to, to my webhost I get a SocketException (0x274c)
"A connection attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because connected host has
failed to respond 38.69.197.71:45623"
I have been debugging this for a long time. 我已经调试了很长时间了。
My app.config has two entries that are relevant for this 我的app.config有两个与此相关的条目
httpWebRequest useUnsafeHeaderParsing="true"
httpRuntime executionTimeout="180"
That helped me through a couple of problems. 这帮助我解决了几个问题。
Now this is my C# code. 现在这是我的C#代码。
HtmlWeb htmlweb = new HtmlWeb();
htmlweb.PreRequest = new HtmlAgilityPack.HtmlWeb.PreRequestHandler(OnPreRequest);
HtmlDocument htmldoc = htmlweb.Load(@"http://www.websitetofetch.com,
"IP", port, "username", "password");
//This is the preRequest config
static bool OnPreRequest(HttpWebRequest request)
{
request.KeepAlive = false;
request.Timeout = 100000;
request.ReadWriteTimeout = 1000000;
request.ProtocolVersion = HttpVersion.Version10;
return true; // ok, go on
}
What am I doing wrong? 我究竟做错了什么? I have enabled the tracer in the appconfig, but I don't get a log on my webhost...? 我在appconfig中启用了跟踪器,但是我没有在我的webhost上登录...?
Log stuff from app.config
<system.diagnostics>
<sources>
<source name="System.ServiceModel.MessageLogging" switchValue="Warning, ActivityTracing" >
<listeners>
<add name="ServiceModelTraceListener"/>
</listeners>
</source>
<source name="System.ServiceModel" switchValue="Verbose,ActivityTracing">
<listeners>
<add name="ServiceModelTraceListener"/>
</listeners>
</source>
<source name="System.Runtime.Serialization" switchValue="Verbose,ActivityTracing">
<listeners>
<add name="ServiceModelTraceListener"/>
</listeners>
</source>
</sources>
<sharedListeners>
<add initializeData="App_tracelog.svclog"
type="System.Diagnostics.XmlWriterTraceListener, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
name="ServiceModelTraceListener" traceOutputOptions="Timestamp"/>
</sharedListeners>
</system.diagnostics>
Can anyone spot the problem I have these setting on and off like a thousand times.. 任何人都可以发现问题我有这些设置开关一千次..
request.KeepAlive = false;
System.Net.ServicePointManager.Expect100Continue = false;
Carl 卡尔
Try downloading the page as a string first, then passing it to HtmlAgilityPack. 尝试先将页面下载为字符串,然后将其传递给HtmlAgilityPack。 This will let you isolate errors that happen during the download process from those that happen during the html parsing process. 这将使您可以将下载过程中发生的错误与html解析过程中发生的错误隔离开来。 If you have an issue with proxybonanza (see end of post) you will be able to isolate that issue from a HtmlAgilityPack configuration issue. 如果您在使用proxybonanza时遇到问题(请参阅帖子末尾),您将能够将该问题与HtmlAgilityPack配置问题隔离开来。
Download page using WebClient: 使用WebClient下载页面:
// Download page
System.Net.WebClient client = new System.Net.WebClient();
client.Proxy = new System.Net.WebProxy("{proxy address and port}");
string html = client.DownloadString("http://example.com");
// Process result
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(html);
If you want more control over the request, use System.Net.HttpWebRequest : 如果您想要更多地控制请求,请使用System.Net.HttpWebRequest :
// Create request
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com/");
// Apply settings (including proxy)
request.Proxy = new WebProxy("{proxy address and port}");
request.KeepAlive = false;
request.Timeout = 100000;
request.ReadWriteTimeout = 1000000;
request.ProtocolVersion = HttpVersion.Version10;
// Get response
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string html = reader.ReadToEnd();
}
catch (WebException)
{
// Handle web exceptions
}
catch (Exception)
{
// Handle other exceptions
}
// Process result
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(html);
Also, ensure that your proxy provider (proxybonanza) allows access from your production environment to your proxies. 此外,请确保您的代理提供程序(proxybonanza)允许从您的生产环境访问您的代理。 Most providers will limit access to the proxies to certain IP addresses. 大多数提供商将限制对某些IP地址的代理的访问。 They may have allowed access to the external IP of the network where you are running locally but NOT the external IP address of your production environment. 他们可能允许访问您在本地运行的网络的外部IP,但不允许访问生产环境的外部IP地址。
It sounds like your web host has disabled outgoing connections from ASP.NET applications for security because it would allow other scripts/apps to perform malicious attacks from their servers. 听起来您的Web主机已禁用ASP.NET应用程序的传出连接以确保安全性,因为它允许其他脚本/应用程序从其服务器执行恶意攻击。
You would have to ask them to unblock connections on your account, but don't be surprised if they say no. 您必须要求他们取消阻止您帐户中的关联,但如果他们拒绝,请不要感到惊讶。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.