获取浏览器呈现的 java 中完全加载的网页源代码

Question

There are few webpage which employs some javascript/ajax calls to fill some fields in webpage during page load or after page load. One example is http://www.sohos.co.uk/MEN-T-Shirts/Iron-Fist-Mens-Clothing/--Iron-Fist-Bloody-Mess-T--Shirt_ct271bd5pd197.html where content in size drop down box is filled by use of javascript.

Is it possible to get fully loaded page of this type in Java? I think, simply using HttpClient or other methods will not suffice but using Selenium, if this can be done, it would be really great.

I want some generic solution which could be applied for all websites.

Any input/pointers would be greatly appreciated.

Thanks

Jitendra

Answer 1

Selenium's WebDriver supports such emulation through different implementations. One of which is HtmlUnit (a headless java browser) - take a look.