how to web scrape autocompleting textfield

Question

I am trying to fill a website form(compareraja.in) to search and compare mobile phones using java. I currently am using the jaunt library, but I just cant get to know, how to fill an autocompleting textfield, what i want is to select a particular item from the autocompleting list after i apply a certain initial letters to the textfield. Is it possible with jsoup or jaunt or htmlunit or any other library?

if yes, which is the most better and easiest choice ? Also how can it be done?

Answer 1

My clue is that first of all you have to retrieve the whole autocompletion list which appears after you've applied a certain letters. If you open the web page in eg Chrome, go to Developer Tools F12 , Network Tab, then you will see that each time you type a letter in the textfield, a corresponding X ML H TTP R equest is logged in the list.

For example, I've typed "htc ":

On the Network Tab the last one XHR Header section contains all the necessary query parameters :

And Response section shows the received data, which is actually being shown in that autocompletion list:

So, you can just make GET XHR to URL http://www.compareraja.in/autocompletedata.ashx?q=htc+&c=mobiles&limit=150 (you can even click this link or paste it to the browser's address line to test), where your URL-encoded initial letters should be placed instead of htc+ . It works fine without timestamp parameter for me.

After that it's easy to parse the response, splitting text by \\n and ; chars, and fill the textfield with selected item.