如何使用Jsoup从网站获取内容

Question

I amm trying to get the data from a website. With this code:

@WebServlet(description = "get content from teamforge", urlPatterns = { "/JsoupEx" })
public class JsoupEx extends HttpServlet {
    private static final long serialVersionUID = 1L;
    private static final String URL = "http://www.moving.com/real-estate/city-profile/results.asp?Zip=60505";

    public JsoupEx() {
        super();
    }

    protected void doGet(HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException {
        Document doc = Jsoup.connect(URL).get();
        for (Element table : doc.select("table.DataTbl")) {
            for (Element row : table.select("tr")) {
                Elements tds = row.select("td");
                if (tds.size() > 1) {
                    System.out.println(tds.get(0).text() + ":"
                            + tds.get(2).text());
                }
            }
        }
    }
}

I am using the jsoup parser. When run, I do not get any errors, just no output.

Please help on this.

Answer 1

With the following code

public class Tester {
    private static final String URL = "http://www.moving.com/real-estate/city-profile/results.asp?Zip=60505";


    public static void main(String[] args) throws IOException {
        Document doc = Jsoup.connect(URL).get();
        System.out.println(doc);

    }

}

I get a java.net.SocketTimeoutException: Read timed out. I think the particuliar URL you are trying to crawl is too slow for Jsoup. Being in Europe, my connection might be slower as yours. However you might want to check for this exception in the log of your AS.

By setting the timeout to 10 seconds, I was able to download and parse the document :

Connection connection = Jsoup.connect(URL);
connection.timeout(10000);
Document doc = connection.get();
System.out.println(doc);

With the rest of your code I get :

Population:78,413

Population Change Since 1990:53.00%

Population Density:6,897

Male:41,137

Female:37,278

.....

Answer 2

thanx Julien, I tried with the following code, getting SocketTimeoutException. And code is

Connection connection=Jsoup.connect("http://www.moving.com/real-estate/city-   
profile/results.asp?Zip=60505");
connection.timeout(10000);
Document doc = connection.get();
System.out.println(doc);