简体   繁体   中英

scraping website that is generated by javascript in C#

I am new to coding. I am trying to make a simple console app that will run internet speed test. I've searched all over and I couldn't find the answer. I tried all the sample answers but I couldn't get the program to run. For now, my program returns 0 which is a value from HTML source document. I need the value from javascript. the website is https://fast.com/en/ I only need the speed test results. I need help. here is my code:

enter code here
class Program
{
    [STAThread]
    static void Main(string[] args) 
    {
        HtmlWeb web = new HtmlWeb();
        string url = "https://fast.com/en/";
        HtmlDocument doc = web.LoadFromBrowser(url, html =>
          {
              return !html.Contains
              ("<div class=\"speed-results-container succeeded\"  
              id=\"speed_value\" ></div>");
          });

         var t1 = doc.DocumentNode.SelectSingleNode
             ("//div[@id='speed-value']").InnerText;
         Console.WriteLine($"{t1}");

    }
}

So the whole "magic" of the test is made in app-ea56f7.js file. This file is sending request and receiving chunks of data from netflix. Unfortunately, as referenced in Running Scripts in HtmlAgilityPack there is no direct way of having this without using a headless browser.

Either use https://www.npmjs.com/package/speedtest-net

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM