C＃使用禁用的JavaScript发送Web请求

Question

I want to download content of one website programatically and it looks like this content is loaded by ajax calls. When I simply disable javascript in my browser, only 1 request is made by this page and all content is loaded without AJAX.

What I need to achieve is to make a web request which will tell web page that I have disabled javascript so it returns me all the content and not just empty body tag with no content at all.

Any suggestions how to do that?

Answer 1

You need to mimic browser.

Steps:

• Use Fiddler and see what is sent by browser.
• Set the same headers/cookies/user agent via C# code.
• If does not work - compare request your code makes with browser's one by using Fiddler as proxy for your C# code (set proxy to http://localhost:8888 )