使用Jsoup从表和网站的所有选项卡中获取链接

Question

I'm new to web scraping so the question may not have been framed perfectly. I am trying to extract all the drug name links from a given page alphbetically and as a result extract all az drug links, then iterate over these links to extract information from within each of these like generic name, brand etc. I have a very basic code below that doesn't work. Some help in approaching this problem will be much appreciated.

public class WebScraper {
  public static void main(String[] args) throws Exception {

    String keyword = "a"; //will iterate through all the alphabets eventually
    String url = "http://www.medindia.net/drug-price/brand-index.asp?alpha=" + keyword; 

    Document doc = Jsoup.connect(url).get();
    Element table = doc.select("table").first();
    Elements links = table.select("a[href]"); // a with href
    for (Element link : links) {
    System.out.println(link.attr("href"));
  }
}

Answer 1

After looking at the website and what you are expecting to get, it looks like you are grabbing the wrong table element. You don't want the first table, you want the second.

To grab a specific table, you can use this:

Element table = doc.select("table").get(1);

This will get the table at index 1, ie the second table in the document.