BlackBerry UTF-8 InputStreamReader on Socket issue

Question

I'm trying to read the response from a server using a socket and the information is UTF-8 encoded. I'm wrapping the InputStream from the socket in an InputStreamReader with the encoding set to "UTF-8".

For some reason it seems like only part of the response is read and then the reading just hangs for about a minute or two and then it finishes. If I set the encoding on the InputStreamReader to "ISO-8859-1" then I can read all of the data right away, but obviously not all of the characters are displayed correctly.

Code looks something like the following

socketConn = (SocketConnection)Connector.open(url);
InputStreamReader is = new InputStreamReader(socketConn.openInputStream(), "UTF-8");

Then I read through the headers and the content. The content is chunked and I read the line with the size of each chunk (convert to decimal from hex) to know how much to read.

I'm not understanding the difference in reading with the two encodings and the effect it can have because it works without issue with ISO-8859-1 and it works eventually with UTF-8, there is just the long delay.

Answer 1

It's hard to get the reason of the delay.

You may try another way of getting the data from the network:

byte[] data = IOUtilities.streamToBytes(socketConn.openInputStream());

I believe the above should be passed without delay. Then having got the bytes from network you can start data processing. Note you can always get a String from bytes representing a string in UTF-8 encoding:

String stringInUTF8 = new String(bytes, "UTF-8");

UPDATE : see the second comment to this post.

Answer 2

I was already removing the chunk sizes on the fly so I ended up doing something somewhat similar to the IOUtilities answer. Instead of using an InputStreamReader I just used the InputStream. InputStream has a read method that can fill an array of bytes, so for each chunk the code looks something like this

byte[] buf = new buf[size];
is.read(buf);
return new String(buf, "UTF-8");

This seems to work, doesn't cause any delays and I can remove the extra information about the chunks on the fly.