使用 Java 中的输入流提取特定路径下的 Json 嵌套对象

Question

Have been searching this for a while, and I've also developed something working but wondering if there is a framework that can make my life easier.

The problem is simple, I have a Json InputStream representing a really large data payload.

I know for sure that this payload contains an array of objects under a known path, and I do not want to parse the stream in memory, instead I would like to seek on the file on the given path and extract all the objects of the nested array one at aa time as a Map<String,Object> .

Example:

{
   "store": {
      "book" : [

         {
           "isbn": "123",
           "author": "author",
           "title": "title",
         },
         ..... many more objects
      ]
   }
}

What I need is to seek on $.store.book and extract individual nested objects for further processing.

I've tried JsonPath (jayway) but the parse method seem to load everything in memory.

I've then used Jackson with the stream library but the solution that I got is a bit convoluted. Is there an easier way to achieve this?

Thanks

Answer 1

One option would be to sequentially look through the data at a limited rate such as 512 or 1024 bytes at a time. You could then parse the bytes to the encoded format which is likely ISO-8859-1 or UTF-8. You could then read all bytes until some delimiter is met, likely ] to signify the end of the array (hopefully). You could then use Jackson or Gson to load just that array of objects.

Answer 2

You need ParamTOFilterBy and FilterValue

If you insist on Using JSONPath the selection would look like this:

JsonPath.read(jsonAsString, "$.store.book[?(@.ParamTOFilterBy==FilterValue)]")

This question talks about filtering a JSON String. Opinion I like Jackson implementation in this answer

Answer 3

If the JSON array you're interested in can be identified by a JSON pointer, then a FilteringParserDelegate with a JsonPointerBasedFilter can do the job. This is basically a streaming parser that skips ahead until the target is found. Then you can continue streaming the interesting tokens, or do data binding.

private static final ObjectMapper mapper = new ObjectMapper();
private static final JsonFactory factory = mapper.getFactory();

public static void processArrayElements(InputStream json, 
                                        JsonPointer pointerToArray,
                                        Consumer<Map<String, Object>> consumer)
    throws IOException {

  JsonParser parser = new FilteringParserDelegate(
      factory.createParser(json),
      new JsonPointerBasedFilter(pointerToArray), false, false);

  if (parser.nextToken() != JsonToken.START_ARRAY) {
    throw new IOException("Expected an array but found " + parser.currentToken());
  }

  while (parser.nextToken() != JsonToken.END_ARRAY) {
    consumer.accept(parser.readValueAs(Map.class));
  }
}

To print the books:

JsonPointer bookArray = JsonPointer.compile("/store/book");
processArrayElements(json, bookArray, System.out::println);