如何使用Avro架构验证JSON？

Question

I'd like to know the feasibility of using an Avro schema to validate JSON that comes into my app. In this post , Doug Cutting suggests using the jsontofrag tool that comes with the avro-tools jar. His example is a trivial one of a JSON "document" that is just a number:

echo 2 | java -jar avro-tools.jar jsontofrag '"int"' - | java -jar avro-tools.jar fragtojson '"int"' - 

While this works, I'd like to know how to do it with a more interesting JSON doc.

When I try this with the example JSON doc and schema on the Avro website it fails, like so:

The Avro schema:

{"namespace": "example.avro",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": ["int", "null"]},
     {"name": "favorite_color", "type": ["string", "null"]}
 ]
}

Example JSON doc

{"name": "Ben",
 "favorite_number": 7,
 "favorite_color": "red"}

But when I try to do it with:

cat user.json | java -jar avro-tools.jar jsontofrag user.avsc - | java -jar avro-tools.jar fragtojson user.avsc -

It get this error (stack trace elided):

Exception in thread "main" org.apache.avro.SchemaParseException: org.codehaus.jackson.JsonParseException: 
Unexpected character ('u' (code 117)): 
expected a valid value (number, String, array, object, 'true', 'false' or 'null') 
at [Source: java.io.StringReader@74dca977; line: 1, column: 2]

Any ideas on how to make this work? Or another way to use an Avro schema to validate JSON?

Answer 1

The usage (and backtrace) for the jsontofrag tool leaves much to be desired; what it means by "schema" is a literal schema string, not a filename containing a schema. (Surprise!) The following tweak to your command worked for me:

cat user.json | java -jar avro-tools.jar jsontofrag "`cat user.avsc`" - | java -jar avro-tools.jar fragtojson "`cat user.avsc`" -

Here I've used old-style backtics for command-substitution; the newer "$(cat user.avsc)" syntax also works in bash and probably in other modern(ish) shells.

Answer 2

fromjson is an alternative to jsontofrag that is perhaps easier and more straightforward.

java -jar avro-tools.jar fromjson --schema-file user.avsc user.json > user.avro

If the JSON is not valid, this will throw an Exception, so that is how one can use this to validate JSON.