C＃.NET-HttpWebRequest-System.Net.WebException-嘗試過多的自動重定向

Question

我的HttpWebRequest類有問題。
我正在嘗試獲取網站的源代碼：
http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
但我總是遇到一個錯誤：

System.Net.WebException occurred
  HResult=-2146233079
  Message=Too many automatic redirections were attempted.
  Source=System
  StackTrace:
       at System.Net.HttpWebRequest.GetResponse()
       at ProjectName.ClassName.MethodName(String urlAddress)
  InnerException: 

那是我的代碼：

Uri uri = new Uri(@"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

我已經使用Fiddler Web調試器工具將Firefox請求與我的C＃.NET請求進行比較，但是仍然沒有答案。

Firefox：

GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive


HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Content-Language: pl-PL
Transfer-Encoding: chunked
Date: Wed, 07 Oct 2015 13:36:31 GMT
X-Cache: HIT from blade110.non.3dart.com
X-Cache-Hits: 116
Server: Apache

C＃.NET：

GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci:+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
Connection: Keep-Alive


HTTP/1.1 301 Moved Permanently
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Language: pl-PL
Location: /film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
Content-Length: 0
Accept-Ranges: bytes
Date: Wed, 07 Oct 2015 13:34:51 GMT
X-Cache: MISS from blade712.non.3dart.com
Server: Apache

我已經閱讀了其他文章，並通過其他方式更新了我的代碼，例如。

request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.TransferEncoding = "gzip, deflate";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0";

request.Referer = "http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983";
request.KeepAlive = true;
request.AllowAutoRedirect = true;
request.MaximumAutomaticRedirections = 250;
request.Proxy = null;
request.UseDefaultCredentials = true;

CookieContainer cookieContainer = new CookieContainer();
request.CookieContainer = cookieContainer;

但沒有任何效果：-/

有人可以幫我解決這個問題嗎？

Answer 1

網站加載時，您需要具有初始Cookie，然后才能獲取深層鏈接。

以下代碼對我有用：

//  cookies
CookieContainer cookieContainer = new CookieContainer();

// make one call to the root of the website
// to get the cookies set
Uri uri = new Uri(@"http://www.filmweb.pl");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

using(var s = response.GetResponseStream())
{
   using(var sr = new StreamReader(s)) 
   {
      // linqpad
      sr.ReadToEnd().Dump(); // to check for errors
   }
}

// we have cookies now
// do the deep link fetch
uri = new Uri(@"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
response = (HttpWebResponse)request.GetResponse();

//store the result
using(var f = File.Create("C:\\temp\\pl.txt"))
{
    response.GetResponseStream().CopyTo(f);
}

確保如果您刮擦網站，則遵守其許可和使用政策。 不要做超出正當使用范圍的事情，也不要做任何侵犯版權的材料。