HTTPURLConnection returning gibberish

Question

I'm modifying some working code to work with a different provider's API (we are switching helpdesk providers).

I am trying to look at the xml coming back to see if I am even on the right track, but all I see coming back is gibberish. I've looked at this question but can't figure out how those answers might apply to my situation.

If I remember correctly, when I used the other API I was able to read the xml coming back in the console here:

while ((line = br.readLine()) != null) {
    System.out.println(line);
}

My question is: Is there a way I can read the stream differently so that I can read the xml that is coming back or do I have another problem?

I'm pretty new to this so any thoughts are appreciated. Further details are below.

Code:

package com.google.gwt.HelpDeskTest.server;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import com.google.gwt.user.server.rpc.RemoteServiceServlet;
import com.google.gwt.HelpDeskTest.client.HelpDeskTestService;
import com.google.gwt.HelpDeskTest.shared.HelpDeskTestException;


@SuppressWarnings("serial")
public class HelpDeskTestImpl extends RemoteServiceServlet implements
    HelpDeskTestService {

    @Override
    public String postToRemoteServer(String serviceUrl)
            throws HelpDeskTestException {
        try {

            final String serverPath= "https://www.myconnectwise.net/v4_6_release/services/system_io/integration_io/processClientAction.rails";

            System.out.println(serverPath);


            final String serverParameters= "<?xml version=%221.0%22 encoding=%22utf-16%22?>" + 
            "<GetTicketAction xmlns:xsi=%22http://www.w3.org/2001/XMLSchema-instance%22 xmlns:xsd=%22http://www.w3.org/2001/XMLSchema%22>" + 
            "<CompanyName>xxxxxx</CompanyName><IntegrationLoginId>xxxxxxx</IntegrationLoginId><IntegrationPassword>xxxxxx</IntegrationPassword>" +
            "<SrServiceRecid>1921</SrServiceRecid></GetTicketAction>";


            System.out.println(serverParameters);

            //Open HttpURLConnection:           

            URL url = new URL(serverPath); 
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();   

            connection.setConnectTimeout(10000); //added this to see if I can address the timeout issue.
            connection.setReadTimeout(10000);

            connection.setDoOutput(true);
            connection.setDoInput(true);
            connection.setInstanceFollowRedirects(false); 
            connection.setRequestMethod("POST"); 
            connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded"); 
            connection.setRequestProperty("charset", "utf-16");
            connection.setRequestProperty("Content-Length", "" + Integer.toString(serverParameters.getBytes().length));
            connection.setUseCaches (false);
            //connection.setChunkedStreamingMode(0);

            DataOutputStream wr = new DataOutputStream(connection.getOutputStream ());
            wr.writeBytes(serverParameters);
            wr.flush();
            wr.close();

            //process response - need to get xml response back.
            //this was the working line of code:
            InputStream stream = connection.getInputStream();

            //put output stream into a string
            BufferedReader br = new BufferedReader(new InputStreamReader(stream));
            String result = "";
            String line;
            while ((line = br.readLine()) != null) {
                System.out.println(line);
                result+= line;
            }

            br.close();
            connection.disconnect();

            System.out.println(result);

            return result;


        }  catch (final Exception e) {
            System.out.println(e.getMessage());
            throw new HelpDeskTestException();
            //handle timeout error

        }
    }   
}

This is the xml I'm attempting to send. I've tested it through the company's API tester and know that it works, and responds by sending xml.

<?xml version="1.0" encoding="utf-16"?>
<GetTicketAction xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <CompanyName>xxxxxx</CompanyName>
    <IntegrationLoginId>xxxxxx</IntegrationLoginId>
    <IntegrationPassword>xxxxx</IntegrationPassword>
    <SrServiceRecid>1921</SrServiceRecid>
</GetTicketAction>

Answer 1

When you are sending the data you specify utf-16 as encoding.

But when you are reading the response you do not specify an encoding, so the default platform encoding is used.

So exchange this line:

BufferedReader br = new BufferedReader(new InputStreamReader(stream));

with this (assuming the response also is encoded in utf-16):

BufferedReader br = new BufferedReader(new InputStreamReader(stream,"utf-16"));

You should actually check the response header to learn which encoding has been used.

Answer 2

So after much searching I found the answer to this. The xml is read as gibberish because it is Gzip compressed. The way to read this is by using the GZIPInputStream. This is because the XML is compressed differently.

HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Accept-Encoding", "gzip");
        InputStreamReader in = new InputStreamReader (new GZIPInputStream(connection.getInputStream()));
        String str;            
        while (true) {
     int ch = in.read();
     if (ch==-1) {
        break;
     }