简体   繁体   中英

get specific data from website using JSOUP in Java?

I am HTML/CSS beginner, so I know little thing about Selector(cssQuery)...I want to get only specific data in the Red Rectangle as in the picture in the link. Please help me, thanks in advance.......

Format:
Title, URL, Place, Content

Here is the code:

public class TestJSOUP {
    public static void main(String[] args) throws Exception{
        Document doc = Jsoup.connect("http://scholarship-positions.com/category/under-graduate-scholarship/").get();

        Elements body = doc.getElementsByTag("a");
        for (Element b : body) {
            System.out.println(b.attr("href"));
            System.out.println(b.text());
        }  
    }        
}

Here is the image- https://i.stack.imgur.com/52Gbb.jpg

This website restrict crawler. So you need to specify UserAgent like below

Document doc = Jsoup.connect("http://scholarship-positions.com/category/under-graduate-scholarship/")
                    .userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36")
                    .get();
Elements es = doc.select("article");
for (Element b : es) {
            String title = b.select("a[href], [title]").get(0).text();
            String url = b.select("a[href]").get(0).attr("href");
            String place = b.select("li.custom-fields-ak-organization").get(0).text();
            String content = b.select("section.entry").get(0).text();
            System.out.println(title + url + place + content);
}

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM