ASP.NET 获取网站的所有页面

Question

I want to parse some websites and get list of all pages on current domain, like a:

sample.com/

sample.com/page1/

sample.com/page2.html

But I can't find samples, how to build this sitemap or tree using C# and ASP.NET

I found only one example: http://www.codeproject.com/Articles/13486/A-Simple-Crawler-Using-C-Sockets

But I can't understand, how author use it

if(Directory.Exists(strUri) == true) { //some code DirectoryInfo dir = new DirectoryInfo(folderName); FileInfo[] fia = dir.GetFiles("*.txt"); }

When I use this code, result in if is always false . When I use only

GetFiles function

I have Exception:

URI formats are not supported

Who have any ideas?

Answer 1

remember that on a Web ambient, you cant read files that way, you need to use

Server.MapPath(url)

to get the physical address to the files, then you can do the loop you are using.