Java InputStream 阻塞读

Question

According to the java api, the InputStream.read() is described as:

If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

I have a while(true) loop doing a read and I always get -1 when nothing's sent over the stream. That's expected.

My question is when would read() ever block? Since if it doesn't get any data it returns -1. I would expect a blocking read to wait until data is received. If you've reached the end of the input stream, shouldn't read() simply wait for data instead of returning -1?

Or does read() only block if there's another thread accessing the stream and your read() cannot access the stream?

---

Which leads me to my next question. I used to have event listener (provided by my library) that would notify me when data is available. When I was notified I would call while((aByte = read()) > -1) store the byte. I was puzzled when I'd get TWO events in very close time proximity and not all my data was being displayed. It seemed like only the tail end of the second event's data would be displayed and the the rest was missing.

I eventually changed my code so that when I get an event I'd called if(inputStream.available() > 0) while((aByte = read()) > -1) store the byte. Now it worked properly and all my data was displayed.

Can someone explain this behavior? The InputStream.available() is said to return the number of bytes you can read before blocking the next caller (of the stream?). Even if I don't use.available() I would expect the read of the first event to just block the read of the second event, but not erase or consume too much stream data. Why would doing this cause not all of my data to be displayed?

Answer 1

The underlying data source for some implementations of InputStream can signal that the end of the stream has been reached, and no more data will be sent. Until this signal is received, read operations on such a stream can block.

For example, an InputStream from a Socket socket will block, rather than returning EOF, until a TCP packet with the FIN flag set is received. When EOF is received from such a stream, you can be assured that all data sent on that socket has been reliably received, and you won't be able to read any more data. (If a blocking read results in an exception, on the other hand, some data may have been lost.)

Other streams, like those from a raw file or serial port, may lack a similar format or protocol to indicate that no more data will be available. Such streams can immediately return EOF (-1) rather than blocking when no data are currently available. In the absence of such a format or protocol, however, you can't be sure when the other side is done sending data.

---

With regard to your second question, it sounds like you may have had a race condition. Without seeing the code in question, I'm guessing that the problem actually lay in your method of "display". Perhaps the attempt to display by the second notification was somehow clobbering the work done during the first notification.

Answer 2

It returns -1 if it's end of stream. If stream is still open (ie socket connection) but no data has reached the reading side (server is slow, networks is slow,...) the read() blocks.

You don't need call available(). I have a hard time understanding your notification design, but you don't need any calls except read() itself. Method available() is there for convenience only.

Answer 3

OK, this is a bit of a mess so first thing lets clear this up: InputStream.read() blocking has nothing to do with multi-threading. If you have multiple threads reading from the same input stream and you trigger two events very close to each other - where each thread is trying to consume an event then you'd get corruption: the first thread to read will get some bytes (possibly all the bytes) and when the second thread gets scheduled it will read the rest of the bytes. If you plan to use a single IO stream in more then one thread, always synchronized() {} on some external constraint.

Second, if you can read from your InputStream until you get -1 and then wait and can read again later, then the InputStream implementation you are using is broken! The contract for InputStream clearly states that an InputStream.read() should only return -1 when there is no more data to read because the end of the stream has been reached and no more data will EVER be available - like when you read from a file and you reach the end ⁽¹⁾ .

The behavior for "no more data is available now, please wait and you'll get more" is for read() to block and not return until there is some data available (or an exception is thrown).

1. As noted deep in the discussion on erickson's (currently top) answer , a FileInputStream implementation can actually read past the "end of file" and provide more data after a read() has returned -1 - if data is added to the file later. This is an edge case and is basically the only such case in common InputStream implementations (or at worst - very very rare). You should take that into account if you know you use FileInputStream and expect the file you read from have additional data added (a common example is tailing a log file), but otherwise it is just a deficiency of the InputStream API and in any case - you'd be better off if you can stop using the java.io style blocking IO APIs and use java.nio non-blocking IO APIs.

Answer 4

By default the behavior of the provided RXTX InputStream is not compliant.

You have to set the receive threshold to 1 and disable the receive timeout:

serialPort.enableReceiveThreshold(1);
serialPort.disableReceiveTimeout();

Source: RXTX serial connection - issue with blocking read()

Answer 5

Aye. Don't give up on your stream yet Jbu. We are talking Serial communication here, For serial stuff, it is absolutely expected that a -1 can/will be returned on reads. yet still expect data at a later time. The problem is that most people are used to dealing with TCP/IP which should always return a 0 unless the TCP/IP disconnected.., then yea. -1 makes sense, However, with Serial there is no data flow for extended periods of time, and no "HTTP Keep Alive", or TCP/IP heartbeat. or (in most cases) no hardware flow control, But the link is physical. and still connected by "copper" and still perfectly live.

Now, if what they are saying is correct, ie: Serial should be closed on a -1, then why do we have to watch for stuff like OnCTS, pmCarroerDetect, onDSR, onRingIndicator, etc... Heck, if 0 means its there, and -1 means its not, then screw all those detection functions: :-)

The problem you may be facing may lay elsewhere.

Now, onto specifics:

A: I'm going to guess that you were in a loop, re-using the same byte[] buffer. 1st message comes in, is not displayed on the screen/log/std out yet (because you are in the loop), then you read the 2nd message, replacing the 1st message data in the buffer. Again, because I'm going to guess that you don't store how much you read, and then made sure to offset your store buffer by the previous read amount.

A: Bravo... thats the good stuff there. Now, you data buffer is inside an IF statement, your 2nd message will not clobber your 1st... well, actually, it was probably just one big(er) message in the 1st place. But now, you will read it all in one shot, keeping the data intact.

A: Ahhh, the good ol' catch all scape goat. The race condition..: ,-) Yes, this may have been a race condition. in fact it may have well been, But. it could also just be the way the RXTX clears the flag. The clearing of the 'data available flag' may not happen as quick as one expects, For example? anyone know the difference between read VS readLine in relation to clearing the buffer the data was previously stored in and re-setting the event flag. Neither do I: .-) Nor can I find the answer yet... but... let me ramble on for a few sentences more. Event driven programming still has some flaws. Let me give you a real world example I had to deal with recently.

• I got some TCP/IP data, lets say, 20 bytes.
• So I receive the OnEvent for Received Data.
• I start my 'read' even on the 20 bytes.
• Before I finish reading my 20 bytes... I get another 10 bytes.
• The TCP/IP however, looks to notify me, oh, sees that the flag is still SET, and will not notify me again.
• However, I finish reading my 20 bytes (available() said there were 20)...
• ... and the last 10 bytes remain in the TCP/IP Q... because I was not notified of them.

See, the notification was missed because the flag was still set... even though I had begun reading the bytes. Had I finished the bytes, then the flag would have been cleared, and I would have received notification for the next 10 bytes.

The exact opposite of what is happening for you now.

So yea, go with an IF available()... do a read of the returned length of data. Then, if you are paranoid, set a timer and call available() again, if there is still data there, then do a read no the new data. If available() returns 0 (or -1), then relax... sit back... and wait for the next OnEvent notification.

Answer 6

InputStream is just an abstract class , unfortunately the implementation decides what happens.

What happens if nothing is found:

• Sockets (ie SocketInputStream ) will block until data is received (by default). But it's possible to set a timeout (see: setSoTimeout ), then the read will block for x ms. If still nothing is received then a SocketTimeoutException will be thrown.

  But with or without timeout, reading from a SocketInputStream can sometimes result in a -1 . ( Eg when multiple clients simultaneously connect to the same host:port , then even though the devices seem connected, the result of a read could immediately restult in a -1 (never returning data). )

• Serialio communication will always return -1 ; You can also set a timeout (use setTimeoutRx ), the read will first block for x ms, but the result will still be -1 if nothing's found. (Remark: but there are multiple serial io classes available, behaviour could be vendor dependent.)

• Files (readers or streams) will result in an EOFException .

Work to a Generic Solution:

• If you wrap any of the above streams in a DataInputStream , then you can use methods like readByte , readChar , etc. All -1 values are converted to EOFException . (PS: If you perform a lot of small reads, then it's a good idea to wrap it in a BufferedInputStream first)
• Both SocketTimeoutException and EOFException extend IOException , and there are several other possible IOException 's. It is convenient to just check for IOException 's to detect communication issues.

Another sensitive topic is flushing. flush in terms of sockets means "send it now", but in terms of Serialio it means "discard the buffer".

Answer 7

I think You can receive the entire data stream if you use thread.sleep()