Java FastXML JSON library: How to parse a nested JSON structure

Question

The following here under is a Java program I am trying out to test JSON Processing using the org.fasterxml.jackson.core and the jackson-databind libraries in a streaming manner.

The goal is to learn how to process JSON and capture the information I want. For now, what I want to accomplish is the following:
1) The JSON that I post here under has a lot of data. The Java program that follows this JSON is my attempt to process this JSON and in particular, capture the "name": US SSN" element under "part1/myAnalysis/matches ." and also "name": MasterCard Credit Card number" , both elements falling under the the scope of " part1/myAnalysis/matches ." ONLY .

Okay, for now, My Goal is: I just want my program to compile and atleast print out those two elements (the ones aforementioned) that interest me .

My attempt at compilation yielded the following results:

Unprocessed property: type
Unprocessed property: incidentTimestamp
Unprocessed property: numOfMatches
Unprocessed property: myReport
Unprocessed property: whatSetItOffEntry
Unprocessed property: seeRestrictedIds
Unprocessed property: status
Unprocessed property: timeStamps
Unprocessed property: count

---

So the JSON, the program is attempting to process is as under. If anyone can point out how to make this program compile and then print out the elements I want. That will be a great 2 step process Task.

 { "type": "ImportantIncidentInfo", "incidentTimestamp": "2014-05-15T10:09:27.989-05:00", "numOfMatches": 4, "myReport": { "docReports": { "part1/.": { "path": [ "unknown" ], "myAnalysis": { "matches": [ { "id": { "major": 1, "minor": 0 }, "name": "US SSN", "position": 13, "string": " 636-12-4567 " }, { "id": { "major": 3, "minor": 0 }, "name": "MasterCard Credit Card Number", "position": 35, "string": " 5424-1813-6924-3685 " } ] }, "cleanedUpData": [ { "startPosition": 0, "endPosition": 65, "frameContent": "" } ], "minedMetadata": { "Content-Encoding": "ISO-8859-1", "Content-Type": "text/html; charset=iso-8859-1" }, "deducedMetadata": { "Content-Type": "text/html; iso-8859-1" } }, "part2/.": { "path": [ "unknown" ], "patternAnalysis": { "matches": [ { "id": { "major": 1, "minor": 0 }, "name": "SSN", "position": 3, "string": " 636-12-4567\\r" }, { "id": { "major": 3, "minor": 0 }, "name": "MasterCard Credit Card Number", "position": 18, "string": "\\n5424-1813-6924-3685\\r" } ] }, "cleanedUpData": [ { "startPosition": 0, "endPosition": 44, "frameContent": "" } ], "minedMetadata": { "Content-Encoding": "windows-1252", "Content-Type": "text/plain; charset=windows-1252" }, "deducedMetadata": { "Content-Type": "text/plain; iso-8859-1" } } } }, "whatSetItOffEntry": { "action": "Log", "component": { "type": "aComponent", "components": [ { "type": "PatternComponent", "patterns": [ 1 ], "not": false } ], "not": false }, "ticketInfo": { "createIncident": true, "tags": [], "seeRestrictedIds": [ { "type": "userGroup", "name": "SiteMasters", "description": "Group for SiteMasters", "masters": [ "04fb02a2bc0fba" ], "members": [], "id": "04fade" } ] }, "letmeknowInfo": { "createNotification": true, "contactNames": [ "someguy@gmail.com" ] } }, "seeRestrictedIds": [ "04fade66c0" ], "status": "New", "timeStamps": [ "2014-03-15T10:09:27.989-05:00" ], "count": 1 } 

Task # 2
2) To process this JSON, I wrote the following Java program.

import com.fasterxml.jackson.databind.*;
import com.fasterxml.jackson.core.*;

import java.io.*;

public class ParseJson {

    public static void main(String[] args) {

        try{

                // TODO Auto-generated method stub
                JsonFactory f = new MappingJsonFactory();
                //JsonParser jp = f.createJsonParser(new File(args[0]));
                JsonParser jp = f.createParser(new File("C:/somepath /in/my/eclipse/project/jsonFormattedModified.json"));


                JsonToken current;

                current = jp.nextToken();
                if (current != JsonToken.START_OBJECT) {
                  System.out.println("Error: root should be object: quiting.");
                  return;
                }

                while (jp.nextToken() != JsonToken.END_OBJECT) {
                  String fieldName = jp.getCurrentName();
                  // move from field name to field value
                  current = jp.nextToken();
                  if (fieldName.equals("matches")) {
                    if (current == JsonToken.START_ARRAY) {
                      // For each of the records in the array
                      while (jp.nextToken() != JsonToken.END_ARRAY) {
                        // read the record into a tree model,
                        // this moves the parsing position to the end of it
                        JsonNode node = jp.readValueAsTree();
                        // And now we have random access to everything in the object
                        System.out.println("Name: " + node.get("name").asText());
                        System.out.println("POS: " + node.get("pos").asText());
                      }
                    } else {
                      System.out.println("Error: records should be an array: skipping.");
                      jp.skipChildren();
                    }
                  } else {
                    System.out.println("Unprocessed property: " + fieldName);
                    jp.skipChildren();
                  }
                }                
              } catch(IOException ie) {
                  ie.printStackTrace();

              } 

        }
}

---

Thanks for all the help.

Answer 1

I suggest you use a very helpful Google API Gson to deal with serialization and deserialization easily. So first of all, create all below classes that match your json structure .

Helper class:

class Helper {
    String type;
    String incidentTimestamp;
    int numOfMatches;
    Report myReport;
    WhatSetItOffEntry whatSetItOffEntry;
    List<String> seeRestrictedIds;
    String status;
    List<String> timeStamps;
    int count;
    //getters and setters
}

Report class:

class Report {
    DocsReport docReports;
    //getters and setters
}

DocsReport class:

class DocsReport {
    @SerializedName("part1/.")
    Part1 part1;
    Part2 part2;
    //getters and setters
}

Part1 class:

class Part1 {
    List<String> path;
    Analysis myAnalysis;
    List<CleanedUpData> cleanedUpData;
    MinedMetadata minedMetadata;
    DeducedMetadata deducedMetadata;
    //getters and setters
}

Analysis class:

class Analysis {
    List<Information> matches;
    //getters and setters
}

Information class:

class Information {
    Identifying id;
    String name;
    int position;
    String string;
    //getters and setters
}

Identifying class:

class Identifying {
    int major;
    int minor;
    //getters and setters
}

CleanedUpData class:

class CleanedUpData {
    int startPosition;
    int endPosition;
    String frameContent;
    //getters and setters
}

MinedMetadata class:

class MinedMetadata {
    @SerializedName("Content-Encoding")
    String contentEncoding;
    @SerializedName("Content-Type")
    String contentType;
    //getters and setters
}

DeducedMetadata class:

class DeducedMetadata {
    @SerializedName("Content-Type")
    String contentType;
    //getters and setters
}

Part2 class:

class Part2 {
    List<String> path;
    Analysis patternAnalysis;
    CleanedUpData cleanedUpData;
    MinedMetadata minedMetadata;
    DeducedMetadata deducedMetadata;
    //getters and setters
}

WhatSetItOffEntry class:

class WhatSetItOffEntry {
    String action;
    Component component;
    TicketInfo ticketInfo;
    LetmeknowInfo letmeknowInfo;
    //getters and setters
}

Component class:

class Component {
    String type;
    List<ComponentData> components;
    Boolean not;
    //getters and setters
}

ComponentData class:

class ComponentData {
    String type;
    List<Integer> patterns;
    Boolean not;
    //getters and setters
}

TicketInfo class:

class TicketInfo {
    Boolean createIncident;
    List<Object> tags;
    List<RestrictedIds> seeRestrictedIds;
    //getters and setters
}

RestrictedIds class:

class RestrictedIds {
    String type;
    String name;
    String description;
    List<String> masters;
    List<Object> members;
    String id;
    //getters and setters
}

LetmeknowInfo class:

class LetmeknowInfo {
    Boolean createNotification;
    List<String> contactNames;
    //getters and setters
}

Then get your two names as follow

Gson gson = new Gson();
Helper data = gson
        .fromJson(
               new BufferedReader(
                        new FileReader(
                                "C:/somepath/in/my/eclipse/project/jsonFormattedModified.json")),
                new TypeToken<Helper>() {
                }.getType());

String name1 = data.getMyReport().getDocReports().getPart1()
        .getMyAnalysis().getMatches().get(0).getName();
String name2 = data.getMyReport().getDocReports().getPart1()
        .getMyAnalysis().getMatches().get(1).getName();

System.out.println(name1+"\n"+name2);

Output:

US SSN
MasterCard Credit Card Number

Answer 2

Two things to consider:

• If you want to keep streaming, calling jp.readValueAsTree() is not a good idea as that will allocate memory to create the tree with all data under the current node (in your code, everything under "matches"), unless you know for sure this is a small tree.
• To get to a value in an array, there is no need to create an array first. The streaming API will let you walk to the value in the array you need.

As @MChaker shows in his answer, if you need most of the data provided in the JSON file, creating a model to hold the data will be beneficial. But if you just need a couple of values, Jackson will let you do that. Allthough, I must admit, getting data in a truly streaming fashion will require some creativity to find ways of keeping track of where you are and what data you expect.

Following code shows the easy way and the streaming way:

import java.io.*;
import java.util.*;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.databind.*;

/**
 * http://stackoverflow.com/q/30288878/3080094
 * Using jackson-databind-2.5.3 which uses
 * jackson-annotations-2.5.0 and
 * jackson-core-2.5.3
 * @author vanOekel
 */
public class JsonTest {

    public static void main(String[] args) {

        try {
            new JsonTest().getNames();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    ObjectMapper jsonMapper = new ObjectMapper();
    JsonFactory jsonFactory = new JsonFactory();

    void getNames() throws Exception {

        final String resourceName = "some.json";
        JsonNode jn;
        try (InputStream in = 
                Thread.currentThread().getContextClassLoader().getResourceAsStream(resourceName)
                ) {
            if (in == null) {
                throw new FileNotFoundException("File not found: " + resourceName);
            }
            jn = jsonMapper.readTree(in);
        } 
        findByPath(jn);

        try (InputStream in = 
                Thread.currentThread().getContextClassLoader().getResourceAsStream(resourceName)
                ) {
            if (in == null) {
                throw new FileNotFoundException("File not found: " + resourceName);
            }
            JsonParser jsonParser = jsonFactory.createParser(in);
            findInStream(jsonParser);
        } 
    }

    final String[] path = new String[] {"myReport", "docReports", "part1/.", "myAnalysis", "matches", "name"};

    void findByPath(JsonNode jn) {

        JsonNode matchesNamesNode = jn;
        for (int i = 0; i < path.length - 1; i++) {
            matchesNamesNode = matchesNamesNode.path(path[i]);
        }
        if (matchesNamesNode.isMissingNode()) {
            throw new RuntimeException("No node with names found.");
        }
        System.out.println("Tree names: " + matchesNamesNode.findValuesAsText(path[path.length - 1]));
    }

    void findInStream(JsonParser jp) throws Exception {

        int pathIndex = 0;
        List<String> names = new ArrayList<String>();
        boolean breakOnClose = false;

        while (jp.nextToken() != null) {
            final String fieldName = jp.getCurrentName();
            if (fieldName == null) {
                continue;
            }
            if (breakOnClose && fieldName.equals(path[path.length - 2])) {
                System.out.println("Stopping search at end of node " + fieldName);
                break;
            }
            if (jp.getCurrentToken() != JsonToken.FIELD_NAME) {
                continue;
            }
            // System.out.println("Field " + fieldName);
            if (pathIndex >= path.length - 1) {
                if (fieldName.equals(path[path.length - 1])) {
                    // move from field name to field value.
                    jp.nextToken(); 
                    String name = jp.getValueAsString();
                    if (name == null) {
                        throw new RuntimeException("No value exists for field " + fieldName);
                    }
                    names.add(name);
                    System.out.println("Found " + fieldName + " value: " + name);
                }
            } else if (fieldName.equals(path[pathIndex])) {
                System.out.println("Found node " + path[pathIndex]);
                pathIndex++;
                if (pathIndex >= path.length - 1) {
                    System.out.println("Looking for names ...");
                    breakOnClose = true;
                    // prevent breaking on "matches" value json-token.
                    jp.nextFieldName(); 
                }
            }
        }
        System.out.println("Streaming names: " + names);
    }

}

Answer 3

  String jsonRecord = "[
                 {
                   a:a,
                   b:
                     {
                      c:
                        [{d:d,e:e},{d:d1,e:e1}]
                     }
                  }
                ]";
 String value ="$.b.c[1].d";
 String str = JsonPath.read(jsonRecord, value);
 system.out.println(str);

it will print d1