BufferedReader can't read all JSON data

Question

I'm using BufferedReader to get JSON data but then I can only display some of it.

When using charset UTF-8 I'm able to display one vehicle data, others just display null.

When using charset ISO-8859-1 I'm able to display data from two vehicles.

Most data comes back with just NULL even though I know it's available. For example I just cannot display information about this vehicle: http://apis.is/car?number=mm202 but I'm always able to display information about this vehicle: http://apis.is/car?number=lj403 .

Here is my AsyncTask to get the JSON data:

protected class GetLicencePlateInfoTask extends AsyncTask<Void, Void, Void> {

    protected String registryNumber, number, factoryNumber, type, subType, registeredAt, status, nextCheck, pollution, weight;

    @Override
    protected Void doInBackground(Void... params) {
        // Ná í bílnúmerið úr ET
        EditText licencePlateET = (EditText) findViewById(R.id.licencePlateEditText);

        // Ná í bílnúmerið
        String licencePlate = licencePlateET.getText().toString();

        // Eyða óþarfa bilum
        licencePlate = licencePlate.trim();

        // Henda út bilum
        licencePlate = licencePlate.replace(" ", "");

        // Setja í uppercase
        licencePlate = licencePlate.toUpperCase();

        // Henda út bandstriki ef það er
        if (licencePlate.contains("-")) {
            licencePlate = licencePlate.replace("-", "");
        }

        // URL + bílnúmer
        String carUrl = "http://apis.is/car?number=" + licencePlate;

        // Virkar eins og browser
        DefaultHttpClient httpClient = new DefaultHttpClient(new BasicHttpParams());

        HttpPost httpPost = new HttpPost(carUrl);
        httpPost.setHeader("Content-type", "application/json");

        InputStream inputStream = null;

        // Geymir JSON gögnin sem við fáum til baka
        String jsonString = null;

        try {
            HttpResponse response = httpClient.execute(httpPost);
            HttpEntity entity = response.getEntity();

            inputStream = entity.getContent();
            // Json er UTF-8
            BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "ISO-8859-1"), 8);
            StringBuilder stringBuilder = new StringBuilder();

            String line = null;
            while ((line = reader.readLine()) != null) {
                stringBuilder.append(line + "\n");
            }
            jsonString = stringBuilder.toString();

            JSONObject jsonObject = new JSONObject(jsonString);
            // Ná í bílnúmer úr results array
            JSONArray resultsArray = jsonObject.getJSONArray("results");

            for (int i = 0; i < resultsArray.length(); i++) {
                try {
                    JSONObject oneObject = resultsArray.getJSONObject(i);
                    // Ná í info úr array
                    registryNumber = oneObject.getString("registryNumber");
                    number = oneObject.getString("number");
                    factoryNumber = oneObject.getString("factoryNumber");
                    type = oneObject.getString("type");
                    subType = oneObject.getString("subType");
                    registeredAt = oneObject.getString("registeredAt");
                    //status = oneObject.getString("status");
                    nextCheck = oneObject.getString("nextCheck");
                    pollution = oneObject.getString("pollution");
                    weight = oneObject.getString("weight");

                } catch (JSONException e) {
                    e.printStackTrace();
                }
            }

            // Log.i("registryNumber", registryNumber);

        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        } catch (ClientProtocolException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (JSONException e) {
            e.printStackTrace();
        } finally {
            try {
                if (inputStream != null) {
                    inputStream.close();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        // Log.i("jsonString", jsonString);

        return null;
    }

    @Override
    protected void onPostExecute(Void aVoid) {
        displayInfoTV.setText(
            "Tegund: " + type + "\n" +
            "Undirtegund: " + subType + "\n" +
            "Skráningarnúmer: " + registryNumber + "\n" +
            "Fastanúmer: " + number + "\n" +
            "Verksmiðjunúmer: " + factoryNumber + "\n" +
            "Fyrst skráð: " + registeredAt + "\n" +
            "CO2 losun: " + pollution + "\n" +
            "Eiginþyngd: " + weight + "\n" +
            //"Staða: " + status + "\n" +
            "Næsta skoðun: " + nextCheck + "\n"
        );
    }
}

Ok I'm getting closer to finding out what's going on.

The StringBuilder comes back with Cannot POST sometimes:

jsonString: Cannot POST /car?number=DD550

But sometimes it works:﹕

jsonString: {"results":[{"registryNumber":"LJ403","number":"LJ403","factoryNumber"...

Any ideas?

Answer 1

When I visit those links using my web browser, the page info says that the charset is "windows-1252".

Try using "windows-1252" or "Cp1252" as the charset name for the InputStreamReader . More generally you can modify your code to extract the charset to use from the response header. (It is in the "Content-Type" header ... and you need to extract it from the media-type string. Here's an example from the HTTP specification.

Content-Type: text/html; charset=ISO-8859-4

If this approach fails, then you should contact the site owner and tell them that either they are sending an incorrect Content-Type header or there is something wrong with their JSON.

Alternatively you could randomly try the other charsets that Java understands. They are listed here: http://docs.oracle.com/javase/1.5.0/docs/guide/intl/encoding.doc.html

Answer 2

use

BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
StringBuilder builder = new StringBuilder();
for (String line = null; (line = reader.readLine()) != null;) {
    builder.append(line).append("\n");
}