[英]how do I use C# to dump a web page's HTML to a text file?
I am working on a project where I need to be able to take a website url www.google.com
for example and get the html for it in a text file to be parsed separately, but I don't know how to do so. 我正在做一个项目,在这个项目中,我需要能够以网站网址
www.google.com
为例,并在单独解析的文本文件中获取它的html,但我不知道该怎么做。
I know there is an easier way to do this than the way I'm doing it, but this is a project aimed at use and knowledge increase. 我知道有一种比我做起来更容易的方法,但这是一个旨在利用和增加知识的项目。
Downloading just a single URL to a file is dead easy using WebClient
: 使用
WebClient
仅将单个URL下载到文件非常简单:
using (var client = new WebClient())
{
client.DownloadFile(url, filename);
}
The trickier bit is that very few web pages really consist of a single piece of HTML - most then load Javascript, or load more data with Javascript, etc. 棘手的一点是,很少有真正由单个HTML组成的网页-大多数网页然后加载Javascript,或使用 Javascript加载更多数据,等等。
In .NET 4.5 and later you might want to use HttpClient
instead of WebClient
- although it's asynchronous and (as far as I can see) doesn't provide anything quite as convenient as DownloadFile
when that's all you want to do. 在.NET 4.5和以后你可能想使用
HttpClient
,而不是WebClient
-虽然这是异步的,(据我可以看到)不提供任何比较一样便利DownloadFile
当这一切你想做的事。
You can try HtmlAgilityPack: 您可以尝试HtmlAgilityPack:
string Url = "http://something";
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(Url);
string contents= doc.DocumentNode.OuterHtml;
File.WriteAllText("X:\abc\def.txt", contents);
C# WebClient class can help you to achieve this: C#WebClient类可以帮助您实现这一目标:
using System;
using System.Net;
using System.IO;
using (WebClient client = new WebClient())
{
string htmlCode = client.DownloadString("http://somesite.com/default.html");
File.WriteAllText(@"c:\YourLocalFolder\somefile.txt", htmlCode);
}
如果还有其他所有文件,则可以使用WebBrowser控件在应用程序中运行IE,这可以运行页面上的jscript等。然后,您可以从C#访问DOM。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.