How to get the attribute “href” from an “a” tag in Jsoup?

Question

I am making a web-scraping project that basically fetches me images from Google Images. In order to get the image src, i use

Element.attr("href")

However, it returns

#

My Code

Document shivWall = Jsoup.connect(searchURL).get();
Elements smallImgElements = shivWall.getElementsByClass("rg_bx rg_di rg_el ivg-i");
smallImgElements.get(0).select("a.rg_l").get(0).attr("href");

I have tried numerous ways, but none of them worked. I even checked again by changing the attr argument to some random value, and it returns null as expected. But, for "href", it just returns a "#". Please help.

Answer 1

In order to get the src attributes of the tags from a website you can try with the below snippet

public class DownloadImages {

    //The url of the website. This is just an example
    private static final String webSiteURL = "http://www.supercars.net/gallery/119513/2841/5.html";

   //The path of the folder that you want to save the images to
   private static final String folderPath = "<FOLDER PATH>";

   public static void main(String[] args) {

   try {
     //Connect to the website and get the html32
     Document doc = Jsoup.connect(webSiteURL).get();

     //Get all elements with img tag ,
     Elements img = doc.getElementsByTag("img");

     for (Element el : img) {
         //for each element get the srs url
         String src = el.absUrl("src");

         System.out.println("Image Found!");

         System.out.println("src attribute is : "+src);

         getImages(src);

   }


} catch (IOException ex) {
   System.err.println("There was an error");
   Logger.getLogger(DownloadImages.class.getName()).log(Level.SEVERE, null, ex);

     }

}

private static void getImages(String src) throws IOException {
     String folder = null;

     //Exctract the name of the image from the src attribute
     int indexname = src.lastIndexOf("/");

     if (indexname == src.length()) {
         src = src.substring(1, indexname);

        }

        indexname = src.lastIndexOf("/");

        String name = src.substring(indexname, src.length());

        System.out.println(name);

        //Open a URL Stream

        URL url = new URL(src);

        InputStream in = url.openStream();

        OutputStream out = new BufferedOutputStream(new FileOutputStream( folderPath+ name));

        for (int b; (b = in.read()) != -1;) {

            out.write(b);

        }

        out.close();

        in.close();

    }

}

This is how you can download images from a website using Jsoup.

Hope this helps.

Answer 2

If you will examine the structure of html that you get with Document shivWall = Jsoup.connect(searchURL).get(); you will find out, that # is the initial value of your href argument. The url are added dynamically with a javascript after page is loaded. Jsoup cannot handle this.

To get urls you can use this code:

for(Element e : shivWall.select(".rg_meta.notranslate")) {
    Gson gson = new Gson();
    Map imageData = gson.fromJson(e.text(), Map.class);
    System.out.println(imageData.get("ou"));
}