web scraping jsoup java unable to scrape full information

Question

I have an information to be scraped from a website. I could scrape it. But not all the information is being scraped. There is so much of data loss. The following images helps you further to understand :

I used Jsoup, connected it to URL and then extracted this particular data using the following code :

Document doc = Jsoup.connect("https://www.awattar.com/tariffs/hourly#").userAgent("Mozilla/17.0").get();
Elements durationCycle = doc.select("g.x.axis g.tick text");

But in the result, I couldn't find any of that related information at all. So I printed the whole document from the URL and it shows the following :

I could see the information when I download the page and read it as an input file but not when I connect directly to URL. But I want to connect it to URL. Is there any suggestion?

I hope my question is understandable. Let me know in case if it is not explanatory.

Answer 1

There is a request body limitation in Jsoup. you should use the maxBodySize parameter:

Document doc = Jsoup.connect("https://www.awattar.com/tariffs/hourly#").userAgent("Mozilla/17.0").maxBodySize(0).get();

"0" is no limit.