Weird encoding issue with AWS ec2 - java

Question

I have a Java Spring Boot application that response locally with nicely encoded text:

Battle Is the Lord's by Rebecca St. James (Ft. Brandon Lake)

But when I run the application on my AWS instance (ec2) it returns:

Battle Is the Lord���s by��Rebecca��St. James (Ft.��Brandon��Lake)

This is my apache HttpClient setup:

...
public class HttpClientHandler {

    public  String getHandler(final String url) {
        return getHandler(url, null);
    }

    public  String getHandler(final String url, final String authHeader) {

        final StringBuilder apiResponse = new StringBuilder();

        final HttpClient client = HttpClientBuilder.create().build();
        log.info("Get url: {}, with authHeader: {}", url, authHeader);
        final HttpGet request = new HttpGet(url);
        request.addHeader("Content-Type", "application/json");

        if (authHeader != null && !authHeader.isEmpty()) {
            request.addHeader("Authorization", authHeader);
        }


        try {
            final HttpResponse response = client.execute(request);
            log.info("Response Code : {}", response.getStatusLine().getStatusCode());
            final BufferedReader rd = new BufferedReader(
                    new InputStreamReader(response.getEntity().getContent()));

            String line;
            while ((line = rd.readLine()) != null) {
                apiResponse.append(line);
            }
        } catch (final Exception e) {
            e.printStackTrace();
        }
        return apiResponse.toString();
    }

}

I am particularly baffled because I see it running fine when I run the application locally.

Thanks for your help,

Rick

Answer 1

The InputStreamReader can accept a charset parameter. If it is not specified then the charset will be determined using the locale on which the JVM runs. When decoding a message it's encoding must have the same format as the input encoding.

The following examples are written in Kotlin but the idea is the same

Converting from a charset to another:

val s = "Battle Is the Lord's by Rebecca St. James (Ft. Brandon Lake)"
val inputStream = s.byteInputStream(Charsets.UTF_8)
val reader = InputStreamReader(inputStream, Charsets.UTF_16)
val lines = BufferedReader(reader).readLines()
println(lines)

will return [??????????]

Reading the data in the same charset will result in a proper read of the data

val s = "your initial string"
val inputStream = s.byteInputStream(Charsets.UTF_8)
val reader = InputStreamReader(inputStream, Charsets.UTF_16)
val lines = BufferedReader(reader).readLines()
println(lines)

will write [your initial string]