从C＃桌面应用程序调用ASP.NET脚本

Question

I'm trying to develop a desktop app to be used as a website scraping tool. My requirement is the user should be able to specify a url in the desktop app.The desktop app should be able to invoke the asp.net script to scrape data from the website and return the records to the desktop app.

Should I use a web service or a ASP.NET runtime for this...???

Any help is appreciated :)

Additional details

The scraping activity is already done.I used HTMLAgility pkg. This is my scraping code to extract a list of company names from a web page.

public static String getPageHTML(String URL)
        {
            String totalCompanies = null;
            HttpWebRequest httpWebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);

            IWebProxy myProxy = httpWebRequest.Proxy;

            if (myProxy != null)
            {
                myProxy.Credentials = CredentialCache.DefaultCredentials;
            }

            httpWebRequest.Method = "GET";

            HttpWebResponse res;

            res = (HttpWebResponse)httpWebRequest.GetResponse();

            HtmlDocument doc1 = new HtmlDocument();

            doc1.Load(res.GetResponseStream());

            HtmlNode node = doc1.DocumentNode.SelectSingleNode("//td[@class='mainbody']/table/tr[last()]/td");

            try
            {
                totalCompanies = node.InnerText;
                return totalCompanies;
            }
            catch (NullReferenceException e)
            {
                   totalCompanies = "No records found";
                    return totalCompanies;

             }




        }

Answer 1

You can use HttpWebRequest within your desktop app, i've done this before (winforms). For example: -

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("url");
var response = new StreamReader(req.GetResponse().GetResponseStream()).ReadToEnd();

You can then use HtmlAgilityPack to parse the data from the response:

 HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
 doc.LoadHtml(response);

 //Sample query
 var node = doc.DocumentNode.Descendants("div")
           .Where(d => d.Attributes.Contains("id")).ToList(); 

Answer 2

(it would be helpful to include more details/be more specific)

If your ASP.NET page already does all the scraping, and all you need to do is access that ASP.NET page, you can simply use HttpWebRequest

http://msdn.microsoft.com/en-us/library/456dfw4f.aspx - short description & tutorial

If that URL is the website TO BE SCRAPED, and you need to include that ASP.NET script in your project, then you need to add it as a web service.

Answer 3

You can do it with both but also you can do it by adding a webbrowser to your desktop application. I don't know why but result is much more faster.