How to force jsoup to wait for a webpage to load completely

Question

The website I am using jsoup to parse loads incrementally. The data I am trying to access loads into the page after a couple of seconds but jsoup only gets everything that was loaded initially.

Is there a way to force jsoup to wait for the page to load completely before attempting to parse it or to build in a delay to allow the page to load completely?

Answer 1

Jsoup can't deal with such a requirement. Jsoup simply interprets HTML (The wait is because of some AJAX call done after page load, since you have mentioned the page loads incrementally).

Alternate would be to use Selenium which can emulate your browser and handle these elegantly.