Jsoup并从表中获取数据

Question

I have tried similar code on the BBC and Wikipedia websites and I can access and handle data from the tables. What is my mistake when I try to do the same on this website? I can get some data with this code but not the specific figures inside the table when I change it to .doc.getElementsByClass("tabela2"); .

    String url = "https://www.itau.com.br/investimentos-previdencia/fundos/rentabilidade-uniclass";
    Document doc = null;
    try {
        doc = Jsoup.connect(url)
        .userAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36")
        .get();
    } catch (IOException e) {
        e.printStackTrace();
    }
    Elements table = doc.getElementsByClass("contentAllInt");
    System.out.println("table: " + table);

Thanks

Answer 1

The website in question uses AJAX to load its content dynamically after initial loading. So it uses JavaScript to render the content.

JSoup does not handle Javascript. It is not a browser after all. So JSoup can't be used here.

2 solutions:

1) Use a real brwoser, eg selenium to access this content in Java. Drawback: You add a new dependency. Real browsers are resource hungry. It may run slowly.

2) Investigate the AJAX calls directly and try to mimic those calls and interpret the results. Usually this is not in HTML format but in JSON, so you may need another interpreting library. I did not look into this option in your case, but usually this is the faster option.