使用 htmlagility 包抓取表格的行项目

Question

I am making program which will show the text from the table.The structure is like that. there are two tables but I want to get text from the 2nd table.

my table data looks like below:

I want to show the first 3 columns of each row of the 2nd table. For this I tried like this.

HtmlWeb web = new HtmlWeb();

HtmlAgilityPack.HtmlDocument doc = web.Load("http://www.banglaeye.com/baby-names/index.php");
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//div[@class='col_box']/table[2]/tr/td");

try
{
    foreach (HtmlNode n in nodes)
    {
        if (k != 0)
        {
            link = n.InnerHtml;
            my_link.Add(link);
            MessageBox.Show(link);               
        }
        k++;
    }
}
catch (NullReferenceException)
{
    MessageBox.Show("No link found");
}

I have found that this URL does a http post method. But Html Agility pack doesn't provide any Http post method. so how can achieve my goal??

Answer 1

The table whose contents you want to grab isn't populated until the user of the browser clicks the "Search" button. If someone were to navigate to that URL normally, they wouldn't see any entries in the table until the button is pressed. That is why HTMLAgilityPack only sees the first row. The button, when clicked, performs an HTTP POST:

letter=All&gender_id=0&origin_id=0&submit=search

Your program must perform this request then load the results into an HtmlDocument using doc.LoadHtml() rather than doc.Load() .

Here are some other Stack Overflow questions you can refer to in order to complete those two tasks:

• HTTP POST
• Getting HtmlDocument from string