Avro Schema与Kafka，ClassCastException？

Question

I've created an Avro schema for records which we are publishing to our Kafka topic. Our actual Kafka record schema is more complex but I've just attached the relevant parts for brevity. We have more than one nested sub class in the record, but for some reason I am getting the following exception when trying to publish the record (Package names have been obscured) :

java.lang.ClassCastException: aaa.bbb.ccc.ddd.Amount cannot be cast to org.apache.avro.generic.IndexedRecord

class KafkaRecord {

    private Amount amount;

    class Amount {

        String currency;
        long value;

    }

}

And this is the current subset from the Avro schema I have defined.

{
  "type" : "record",
  "name" : "KafkaRecord",
  "namespace" : "com.company.department",
  "fields" : [ {
    "name" : "amount",
    "type" : {
      "type" : "record",
      "name" : "Amount",
      "namespace" : "aaa.bbb.ccc.ddd",
      "fields" : [ {
        "name" : "value",
        "type" : "long"
      }, {
        "name" : "currency",
        "type" : "string"
      } ]
    }
  }
}

Our JSON representation of the object (KafkaRecord) looks like so:

{
  "amount": {
    "currency": "GBP",
    "value": 12345
  }
}

I can't seem to figure out why Avro doesn't like this nested record, and I'd prefer not to strip apart these nested classes as it would make the JSON record very difficult to read and hard to manage the classes.

If anyone is able to point out what I am doing wrong here that would be great!

Answer 1

Well, you don't need to write your own Java files. Sounds like you did, and so the error is as it says - Amount class is not an IndexedRecord

For example, if I take your schema and run

java -jar ~/Downloads/avro-tools-1.8.2.jar compile schema KafkaRecord.avsc .

Then look at the file, we see that it extends some Avro java classes.

$ head -n 15 aaa/bbb/ccc/ddd/Amount.java
/**
 * Autogenerated by Avro
 *
 * DO NOT EDIT DIRECTLY
 */
package aaa.bbb.ccc.ddd;

import org.apache.avro.specific.SpecificData;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;

@SuppressWarnings("all")
@org.apache.avro.specific.AvroGenerated
public class Amount extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {

The way to do this programmatically is documented by the Avro Maven Plugin guide.

And yes, Avro works fine with nested records, and personally I like using IDL to more easily create them

@namespace("com.company.department")
protocol KafkaEventProtocol {
  @namespace("aaa.bbb.ccc.ddd")
  record Amount {
    string currency;
    long value;
  }

  record KafkaValue {
    Amount amount;
  }
}