Java and HttpUnit having GZip bugs

Question

I've been trying to find workarounds to this for nearly 6 straight hours now but to no avail.
I have a simple HttpUnit program that does some sign in to a website. The problem is that I'm getting this error when I finally submit the log in form:

Exception in thread "main" java.io.EOFException: Unexpected end of ZLIB input stream
   at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:223)
   at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:141)
   at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
   at com.meterware.httpunit.WebResponse.readFromStream(WebResponse.java:967)
                         ..and so on..  

It appears a bug in the way JDK was managing the gzip file was reported back in 2002 (!). It appears Java still has this bug and the only workaround is to tell HttpUnit to not accept gzip encoded data, like so:

WebConversation.getClientProperties().setAcceptGzip(false);

Unfortuantely this I cannot do. For the sign in to be successful, I need to have gzip enabled, otherwise I'm just redirected to the generic home page (without being signed in).
I suspect the only way to make this work would be to change the source code of HttpUnit and handle the EOF excpetion in it's gzip parsing. I desperately need help. Does anyone have a solution to this?

If not, then could you suggest some equivalently simple API in Java, such as HttpUnit, for tasks such as filling forms and signing in.

Answer 1

I suspect the only way to make this work would be to change the source code of HttpUnit and handle the EOF excpetion in it's gzip parsing.

This is often a solution for developers when there are bugs in third party libraries. If you know what the problem is in HttpUnit, why not fix it locally? You can even push the fix back to the project and hope they adopt it.

If not, then could you suggest some equivalently simple API in Java, such as HttpUnit, for tasks such as filling forms and signing in.

If you mean a library to traverse websites, there are plenty. If you want a "headless" HTTP client, try Apache HttpComponents HttpClient. If you want something that actually uses a browser, try Selenium .

Answer 2

It seems this problem was fixed in HttpUnit 1.7 (I was using 1.6).
Anyways, I'm shifting to HttpComponents as suggested by @ShaggyFrog since now I'm running into bigger, unknown problems with cookies and sessions.

Edit: It seems HttpComponents was overly complicated. At one point I had to manually handle Http 302 redirects. So I decided to shift to HtmlUnit, which is extremely similar to HttpUnit but without any cookie or gzip problems. Worked like a charm.