How to remove key value pairs from a JSON file in java

Question

I was wondering if anyone can help me or hint me towards how to edit the attached dummy JSON file in Java.

As you can see I have a head object that contains many values and children that follow the same pattern.

I wanted to know if there was a way to remove all the keys where the value is -1.

Following is what I was trying based on many websites using jackson :

try {
            // create object mapper instance
            ObjectMapper mapper = new ObjectMapper();

            // convert JSON file to map
            Map<?, ?> map = mapper.readValue(Paths.get("test.json").toFile(), Map.class);

            // print map entries
            for (Map.Entry<?, ?> entry : map.entrySet()) {
                isInteger = main.isObjectInteger(entry.getValue());

                
//              System.out.println("if value is all: " + entry.getKey() + "=" + entry.getValue());
//              

The above code will display the structure of the file, however my problem is reaching the -1 values inside the children and removing them.

Using the.getClass and.simpleName methods I know that it is an arrayList but I am confused as to how to search through it.

Any help will be greatly appreciated!

Answer 1

There are two main techniques to parse and generate JSON data (as well as many other formats like XML etc): object mapping and event/token/stream-oriented processing. The second way is the best way for many cases, including filtering. Props:

• the file/data doesn't require to be loaded entirely into memory, you can process megs/gigs with no problems
• it works much more faster, especially for large files
• it's easy to implement any custom type/rule of transformation with this pattern

Both Gson and Jackson supports stream-oriented processing. To illustrate the idea here is just an example using a tiny parser/generator https://github.com/anatolygudkov/green-jelly

import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonBufferedWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;

import java.io.StringWriter;

public class UpdateMyJson {
    private static final String jsonToUpdate = "{\n" +
            "\"name\": \"Head\",\n" +
            "\"missed\": -1,\n" +
            "\"children\": [\n" +
            "    {\n" +
            "        \"name\": \"project1\",\n" +
            "        \"fixes\": 0,\n" +
            "        \"commits\": -1,\n" +
            "    },\n" +
            "    {\n" +
            "        \"name\": \"project2\",\n" +
            "        \"fixes\": 20,\n" +
            "        \"commits\": 5,\n" +
            "    }\n" +
            "]\n" +
            "}";

    public static void main(String[] args) {
        final StringWriter result = new StringWriter(); // you can use FileWriter

        final JsonParser parser = new JsonParser();
        parser.setListener(new MyJsonUpdater(new AppendableWriter<>(result)));
        parser.parseAndEoj(jsonToUpdate); // if you read a file with a buffer,
        // to don't load the whole file into memory,
        // call parse() several times (part by part) in a loop until EOF
        // and then call .eoj()

        System.out.println(result);
    }

    static class MyJsonUpdater extends JsonEventPump {
        MyJsonUpdater(final JsonBufferedWriter output) {
            super(output);
        }

        @Override
        public boolean onNumberValue(final JsonNumber number) {
            if (number.mantissa() == -1 && number.exp() == 0) {
                return true; // return immediately
            }
            return super.onNumberValue(number); // otherwise pump the value to the result JSON
        }
    }
}

Answer 2

In Jackson you can read whole JSON payload as JsonNode and iterate over all properties check given condition. In case condition is met you can remove given field. To do that you need to implement recursive method . Take a look on below example:

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import java.util.Objects;
import java.util.function.Predicate;

public class JsonRemoveSomeFieldsApp {

    public static void main(String[] args) throws IOException {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.INDENT_OUTPUT);
        JsonNode root = mapper.readTree(jsonFile);

        JsonCleaner jsonCleaner = new JsonCleaner(root, (node) -> node.isNumber() && node.numberValue().intValue() == -1);
        JsonNode result = jsonCleaner.removeAll();

        // write to file
        mapper.writeValue(System.out, result);
    }
}

class JsonCleaner {

    private final JsonNode root;
    private final Predicate<JsonNode> toRemoveCondition;

    JsonCleaner(JsonNode node, Predicate<JsonNode> toRemoveCondition) {
        this.root = Objects.requireNonNull(node);
        this.toRemoveCondition = Objects.requireNonNull(toRemoveCondition);
    }

    public JsonNode removeAll() {
        process(root);
        return root;
    }

    private void process(JsonNode node) {
        if (node.isObject()) {
            ObjectNode object = (ObjectNode) node;
            Iterator<Map.Entry<String, JsonNode>> fields = object.fields();
            while (fields.hasNext()) {
                Map.Entry<String, JsonNode> field = fields.next();
                JsonNode valueToCheck = field.getValue();
                if (valueToCheck.isContainerNode()) {
                    process(valueToCheck);
                } else if (toRemoveCondition.test(valueToCheck)) {
                    fields.remove();
                }
            }
        } else if (node.isArray()) {
            ArrayNode array = (ArrayNode) node;
            array.elements().forEachRemaining(this::process);
        }
    }
}

For below JSON payload:

{
  "name": "Head",
  "missed": -1,
  "covered": -1,
  "children": [
    {
      "name": "project1",
      "missed": -1,
      "covered": -1,
      "children": [
        {
          "name": "project1",
          "missed": 10,
          "covered": 11
        }
      ]
    },
    {
      "name": "project1",
      "missed": -1,
      "covered": 12,
      "children": [
        {
          "name": "project1",
          "missed": 10,
          "covered": -1
        }
      ]
    }
  ]
}

above code prints:

{
  "name" : "Head",
  "children" : [ {
    "name" : "project1",
    "children" : [ {
      "name" : "project1",
      "missed" : 10,
      "covered" : 11
    } ]
  }, {
    "name" : "project1",
    "covered" : 12,
    "children" : [ {
      "name" : "project1",
      "missed" : 10
    } ]
  } ]
}

See also:

• Flattening a 3 level nested JSON string in java