Java socket InputStream.read() not behaving as expected

Question

Ive read many tutorials and posts about the java InputStream and reading data. Ive established a client and server implementation but i'm having weird issues where reading a variable length "payload" from the client is not consistent.

What im trying to do is to transfer up 100kB max in one single logical payload. Now i have verified that the TCP stack is not sending one mahousive 100kB packet from the client. I have played about with different read forms as per previous questions about the InputStream reading but ive nearly teared my hair out trying to get it to dump the correct data.

Lets for example say the client is sending a 70k payload.

Now the first observation i've noticed is that if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[]. When free running the byte[] will be different sizes every time i run the code with practically the same payload.

Timing problems?

second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour ie i get the correct payload size.

please understand i dont like the way ive had to get this to work and im not happy with the solution.

What experiences, problems have you had/seen respectively that might help me fix this code to be more reliable, easier to read.

    public void listenForResponses() {
    isActive = true;
    try {
        // apprently read() doesnt return -1 on socket based streams
        // if big stuff comes through, TCP packets are segmented, but the inputstream 
        // does something odd and doesnt return the correct raw data.
        // this is a work around to accept vari-length payloads into one byte[] buffer
        byte[] inBuffer = new byte[1];
        byte[] buffer = null;
        int bytesRead = 0;

        byte[] finalbuffer = new byte[0];

        boolean isMultichunk = false;

        InputStream istrm = currentSession.getInputStream();

        while ((bytesRead = istrm.read(inBuffer)) > -1 && isActive) {
            buffer = new byte[bytesRead];
            buffer = Arrays.copyOfRange(inBuffer, 0, bytesRead);

            int available = istrm.available();

            if(available < 1) {
                if(!isMultichunk) {
                    finalbuffer = buffer;
                }
                else {
                    finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
                }
                notifyOfResponse(deserializePayload(finalbuffer));
                finalbuffer = new byte[0];
                isMultichunk = false;
            }
            else {
                if(!isMultichunk) {
                    isMultichunk = true;
                    finalbuffer = new byte[0];
                }
                finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
            }

        }
    } catch (IOException e) {
        Logger.consoleOut("PayloadReadThread: " + e.getMessage());
        currentSession = null;
    }
}

Answer 1

InputStream is working as designed.

if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[].

That's because stepping through the code is slower, so more data drives between reads, enough to fill your buffer.

When free running the byte[] will be different sizes every time i run the code with practically the same payload.

That's because InputStream.read() is contracted to block until at least one byte has been transferred, or EOS or an exception occurs. See the Javadoc. There's nothing in there about filling the buffer.

second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour ie i get the correct payload size.

That's the correct behaviour in the case of a 1 byte buffer for exactly the same reason given above. It's not the 'correct behaviour' for any other size.

NB Your copy loop is nonsense. available() has few correct uses, and this isn't one of them.

while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

NB (2) read() does indeed return -1 on socket-based streams, but only when the peer has shutdown or closed the connection.