Capricious Json parsing errors in Java

Question

I have a Java server app with an endpoint that parses Json. I make requests with Wget like this,

wget --header="Content-Type: application/json" --post-file=somefile.json %SERVER_URL%

The file is parsed like this,

InputStream in = httpExchange.getRequestBody();
JsonReader reader = Json.createReader(in);

Create reader throws an exception from time to time when the json file is big (around 10,000 lines), but it does succeed at times for the same file. The error messages look like this,

Unexpected char -1 at (line no=4029, column no=228, offset=204873)

If I go to that line in the file, it looks fine. I also tried parsing the file in Javascript, simply by adding 'var a = ' to the beginning of the file and it's perfectly fine. I've tried bisecting the file, but then I realized that it could sometimes succeed as the file got smaller, regardless the contents...

Could it be that Java starts streaming the file before wget has completely posted it or something weird like that? And the parser sees premature EOFs or something...

Clueless :(

Edit:

I've saved the file to a String first, with this function,

static String convertStreamToString(java.io.InputStream is) {
    java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\\A");
    return s.hasNext() ? s.next() : "";
}

then I replaced the JsonReader creation with this,

InputStream in = he.getRequestBody();
String reqBodyString = convertStreamToString(in);
JsonReader reader = Json.createReader(new StringReader(reqBodyString));

With NetBeans, I saved the value of reqBodyString to a file and the file ends prematurely. It looks I'm not getting the whole body... Something must be closing the stream too quickly... ? :(

Answer 1

The errors were caused by the client finishing the connection too quickly while the server is still reading the stream.

HttpExchange returns a FixedLengthInputStream. In the debugger, I could see the stream's private member "remaining" was still +100KB even though apparently it had finished reading. For reference, here's the implementation of FixedLengthInputStream .

I've added this message to the server,

        try{
            JsonReader reader = Json.createReader(in);
            // ... do stuff
        }
        catch(JsonParsingException exception) {
            String msg = "JsonParsingException: " + exception.getLocalizedMessage();
            msg += " (could be caused by a premature EOF if the client timed out too quickly)";
            logMessage(msg);                
        }

The solution for the client would be to increase the timeout,

wget --timeout=30 ...

However, this doesn't seem to work in wget for Windows,

> wget64 --version
GNU Wget 1.17.1 built on mingw32.

I installed wget on cygwin and that one works even without the timeout parameter,

$ wget --version
GNU Wget 1.18 built on cygwin.