How to extract <a> from google scholar that contains link to download pdf file

Question

I need to extract the tag from html of google scholar. I've written the script but it extracts all the 's. and I cant find any way to extract the specific tag where the download link of the paper is resting. Please Help.! Below is the code

  public static void main(String[] args) throws IOException {
 Document doc;
        try {


            doc = Jsoup.connect("https://scholar.google.com.pk/scholar?q=Bergmark%2C+D.+%282000%29.+Automatic+extraction+of+reference+linking+information+from+online+documents.+Technical+Report+CSTR2000-1821%2C+Cornell+Digital+Library+Research+Group&btnG=&hl=en&as_sdt=0%2C5").get();


            String title = doc.title();
            System.out.println("title : " + title);


            Elements links = doc.select("a[href]");
        // Elements link = doc.select(".pdf");
            for (Element link : links) {

                // get the value from href attribute
                System.out.println("\nlink : " + link.attr("href"));
               System.out.println("text : " + link.text());

            }

        } catch (IOException e) {
            e.printStackTrace();
        }

And here this is the structure of this tag :

<a href="https://ecommons.cornell.edu/bitstream/handle/1813/5809/2000-1821.pdf?sequence=1" data-clk="hl=en&amp;sa=T&amp;oi=gga&amp;ct=gga&amp;cd=0&amp;ei=YBMXWYbRO8a72Ab_2o24CQ"><span class="gs_ctg2">[PDF]</span> cornell.edu</a>

Answer 1

Use div.gs_ggsd and a[href] as css query

Here

div.gs_ggsd => Select all the div Tag that have class name gs_ggsd

Example :

try {
    Document doc = Jsoup
            .connect("https://scholar.google.com.pk/scholar?q=Bangla+Speech+Recognition")
            .userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36")
            .get();

    String title = doc.title();
    System.out.println("title : " + title);

    Elements links = doc.select("div.gs_ggsd").select("a[href]");

    for (Element link : links) {
        System.out.println("\nlink : " + link.attr("href"));
        System.out.println("text : " + link.text());
    }

} catch (IOException e) {
    e.printStackTrace();
}

Read More : https://jsoup.org/cookbook/extracting-data/selector-syntax