如何从Yahoo搜索中提取结果？

Question

I need to pass all the results retrieved by a yahoo search with an URL like that:

https://es.search.yahoo.com/search?p=madrid&fr=yfp-t-777

and display them like this.

Answer 1

You can scrape data from yahoo using Jsoup
Here is the code using Jsoup to parse all the result from yahoo search with the URL you give

public static void main(String[] args) throws IOException {
        String url = "https://es.search.yahoo.com/search?p=madrid&fr=yfp-t-777";
        while (true) {
            System.out.println("Getting data from " + url);
            Document doc = Jsoup.connect(url).timeout(10000).userAgent("Mozilla/5.0").get();
            Elements sections = doc.select("ol.searchCenterMiddle").first().select("div.options-toggle");
            if (sections.isEmpty()) {
                break;
            }
            for (Element section : sections) {
                try {
                    System.out.println(section.getElementsByTag("a").first().text());
                    System.out.println(section.getElementsByTag("span").first().text() + " " + section.select("a.tri").first().text());
                    System.out.println();
                } catch (Exception e) {
                }
            }
            url = doc.select("a.next[href]").attr("href");
        }
}