Java HttpURLConnection编码错误

Question

Trying to read a generated XML from a MS Webservice

URL page = new URL(address);
StringBuffer text = new StringBuffer();
HttpURLConnection conn = (HttpURLConnection) page.openConnection();
conn.connect();
InputStreamReader in = new InputStreamReader((InputStream) conn.getContent());
BufferedReader buff = new BufferedReader(in);
box.setText("Getting data ...");
String line;
do {
  line = buff.readLine();
  text.append(line + "\n");
} while (line != null);
box.setText(text.toString());

or

URL u = new URL(address);
URLConnection uc = u.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {

    inputLine = java.net.URLDecoder.decode(inputLine, "UTF-8");
  System.out.println(inputLine);
}
in.close();

Any page reads fine except the web service output it reads the greater and less than signs strangely

it read < to "& lt;" and > to "& gt;" without spaces, but if i type them here without spaces stackoverflow makes them < and >

Please help thanks

Answer 1

First there seem to be a confusion on this row:

inputLine = java.net.URLDecoder.decode(inputLine, "UTF-8");

This effectively says that you expect every row in the document that your server is providing to be URL encoded. URL encoding is not the same as document encoding.

http://en.wikipedia.org/wiki/Percent-encoding

http://en.wikipedia.org/wiki/Character_encoding

Looking at your code snippet, I think URL encoding (percent encoding) is not what you're after.

In terms of character encoding. You are making a conversion on this line:

InputStreamReader in = new InputStreamReader((InputStream) conn.getContent());

conn.getContent() returns an InputStream that operates on bytes, whilst the reader operates on chars - the character encoding conversion is done here. Checkout the other constructors of InputStreamReader which takes the encoding as second argument. Without the second argument you are falling back on whatever is your platform default in java.

InputStreamReader(InputStream in, String charsetName)

for instance lets you change your code to:

InputStreamReader in = new InputStreamReader((InputStream) conn.getContent(), "utf-8");

But the real question will be "what encoding is your server providing the content in?" If you own the server code too, you may just hard code it to something reasonable such as utf-8 . But if it can vary, you need to look at the http header Content-Type to figure it out.

String contentType = conn.getHeaderField("Content-Type");

The contents of contentType will look like

text/plain; charset=utf-8

A short hand way of getting this field is:

String contentEncoding = conn.getContentEncoding();

Notice that it's entirely possible that no charset is provided, or no Content-Type header, in which case you must fall back on reasonable defaults.

Answer 2

Mark Rotteveel is correct, the webservice is the culprit here it's for some reason sending the greater than and less than sign with the & lt and & gt format

Thanks Martin Algesten but i have already stated i worked around it i was just looking for why it was this way.