如何根据 Avro 模式验证 JSON

Question

I have a JSON response from an API and I would like to validate against an existing Avro schema(strict validation with names and types).

The response is of type

{"name":"alex","age":23,"sex":"M","active":"true"}

The schema has the above types with the data types and I would like to validate the schema and throw an exception in case it fails.(Preferably JAVA).

I have read a solution using the command line but I wanted to do it programmatically.

Thanks in advance

Answer 1

This is how you can validate it programatically.

import org.apache.avro.AvroTypeException;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.Decoder;
import org.apache.avro.io.DecoderFactory;

import java.io.*;

public class MainClass {
    public static void main (String [] args) throws Exception {
        Schema schema = new Schema.Parser().parse("{\n" +
                "     \"type\": \"record\",\n" +
                "     \"namespace\": \"com.acme\",\n" +
                "     \"name\": \"Test\",\n" +
                "     \"fields\": [\n" +
                "       { \"name\": \"name\", \"type\": \"string\" },\n" +
                "       { \"name\": \"age\", \"type\": \"int\" },\n" +
                "       { \"name\": \"sex\", \"type\": \"string\" },\n" +
                "       { \"name\": \"active\", \"type\": \"boolean\" }\n" +
                "     ]\n" +
                "}");
        String json = "{\"name\":\"alex\",\"age\":23,\"sex\":\"M\",\"active\":true}";
        System.out.println(validateJson(json, schema));
        String invalidJson = "{\"name\":\"alex\",\"age\":23,\"sex\":\"M\"}"; // missing active field
        System.out.println(validateJson(invalidJson, schema));
    }

    public static boolean validateJson(String json, Schema schema) throws Exception {
        InputStream input = new ByteArrayInputStream(json.getBytes());
        DataInputStream din = new DataInputStream(input);

        try {
            DatumReader reader = new GenericDatumReader(schema);
            Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
            reader.read(null, decoder);
            return true;
        } catch (AvroTypeException e) {
            System.out.println(e.getMessage());
            return false;
        }
    }
}

Answer 2

NOTE: If you are on a java project and facing above error during validation of json.

In our project we expect a very long JSON object with multiple optional fields, so it was hard to find where is it failing.

Tried different ways using SpecificDatumReader/GenericDatumReaders and other probable ways, but I was always ending up with Expected start-union. Got VALUE_STRING

We have a SpringBoot application and we were already converting the avsc files to Java. So the idea was if we are able to populate the Class object and able to serialize it successfully, it shows that JSON received is valid.

So we ended up using a basic ObjectMapper to populate the class. And if you are compiling avsc files to Java using avro library, then it comes with some default methods to encode/decode the class

ObjectMapper mapper = new ObjectMapper();
// DeSerializing the JSON to Avro class, but this doesn't check for Schema restrictions
StatusUpdated obj = mapper.readValue(payload, StatusUpdated.class);
// Encoding the class and serializing to raw format, this step validates based on schema
obj.toByteBuffer();