使用HtmlUnit和Apache发送请求，同时维护cookie

Question

I want to write a class that uses both Apache 4.x lib and HtmlUnit. Essentially, I need the some requests to behave like a real browser, which HtmlUnit accomplishes nicely. The downside of HtmlUnit is that it's very slow, and many of my requests don't require it.

I know that disabling css/javascript in HtmlUnit can increase processing speed but it's not enough. I want to implement both HtmlUnit and Apache (or anything else that can do the job).

The only requirement I have is having the ability to define headers and pass cookies.

So, is it possible to, for example:

• Send a request to Website.com using Apache library and collect the cookies.
• Send a request to Website.com/Page2 using HtmlUnit while maintaining the session and executing the javascript on page load.

So my questions are.. Can I accomplish what I'm trying to do? Or if I'm going about it the wrong way, is there a method to drastically speed up requests using HtmlUnit? Can I share the CookieContainer between HtmlUnit's WebClient and Apache's HttpClient ?

Answer 1

Yes, this is possible. I also do this sometimes. You need to extract the cookies from htmlunit before using httpclient. You do that by calling

Set<Cookie> cookies = webClient.getCookieManager().getCookies();

You can loop through the set and extract whatever cookie you need. When using httpclient pass the cookie as your header parameter. It will work.