how to interact with website using HttpURLConnection?

Question

Here is the story: There is a website with some form fields. After filling in the fields the user can click on search. By clicking the search button a list of links will be returned and shown underneath the form. If the number of links is more than 10 the user can click on next which is a javascript button .

A part of a java program of mine connects to this website using HttpUrlConnection and sends the form data through the established connection. Retrieving the first 10 links is not a problem ; I mean the connection with website, sending the field data, and getting the first 10 links is not a problem at all. My problem is how to get the next 10 links.

Thanks for helping.

Answer 1

Take a look to HtmlUnit . This frameworks behaves like a browser and allows you to navigate programmatically between pages at your will. It executes Javascript embedded in pages, too.