简体   繁体   中英

Populating nested avro schema

Here is the schema :

    {
      "type" : "record",
      "name" : "avro_schema",
      "namespace" : "avro",
      "doc" : "The generic avro schema used by publishers to publish events 
      "fields" : [ 

          {"name" : "event", 
                        "type" : {
                             "type" : "record",
                             "name" : "event_meta_data",
                             "fields" : [
                                {"name" : "event_name", 
                                 "type" : "string", 
                                 "doc" : "The name of the event.  In the CDC, this field is populated with the name of the data base table or segment."},

                                {"name" : "operation_type", 
                                 "type" : "string", 
                                "doc": "The operation or action that triggered the event.  e.g., Insert, Update, Delete, etc."},

                                {"name" : "transaction_identifier", 
                                 "type" : ["string", "null"], 
                                 "default" : "NONE",
                                "doc" : "A unique identifier that identifies a unit or work or transaction.  Useful in relating multiple events together."},

                                {"name" : "event_publication_timestamp_millis", 
                                 "type" : "string", 
                                "doc": "timestamp when the event was published"},

                                {"name" : "event_publisher", 
                                 "type" : "string", 
                                "doc" : "The system or application that published the event"},

                                {"name" : "event_publisher_identity",
                                "type": ["string", "null"],
                                "default" : "NONE",
                                "doc": "The identity (user) of the system or application that published the event"},

                                 {"name" : "event_timestamp_millis", 
                                 "type" : ["string", "null"],
                                 "default" : "NONE",
                                "doc": "timestamp when the event occured"},

                                {"name": "event_initiator",
                                "type": ["string", "null"],
                                "default" : "NONE",
                                "doc" : "The system or application that initiated the event"},

                                {"name": "event_initiator_identity",
                                "type" : ["string", "null"],
                                "default" : "NONE",
                                "doc": "The system id or application id that initiated the event" }


                              ]},
                              "doc" : "The data about the published event"
          },
          { "name" : "contents",
                        "type" : {
                            "name": "data_field_groups",
                            "type": "array",
                            "items": {
                                "type": "record",
                                "name": "data_field_group",
                                "fields" : [
                                    {"name": "data_group_name",
                                     "type": "string"
                                    },
                                    {
                                    "name": "data_fields",
                                    "type": {
                                        "type": "array",
                                        "items": {
                                            "name": "data_field",
                                            "type":  "record",
                                            "fields":[
                                                {"name" : "data_field_name",
                                                  "type" : "string",
                                                  "doc" : "The field name"},  

                                                {"name": "data_field_type",
                                                "type": "string",
                                                "doc" : "The data type is one of the following values: string, boolean, int, long, float, double or bytes"},

                                                {"name" : "data_field_value",
                                                 "type" : ["string", "boolean", "int", "long", "float", "double","bytes", "null"],
                                                  "doc" : "The value"}
                                            ]
                                        }

                                    }

                                }
                            ]
                        }
                        },
                        "doc" : "The datafields for the for the published event"
          } 
      ]
    }

            Schema schema = new Schema.Parser().parse(new File("resources/v2.avsc"));


    GenericRecord AvroRec = new GenericData.Record(schema);
            GenericRecord event = new GenericData.Record(schema.getField("event").schema());
            //GenericRecord contents = new GenericData.Record(schema.getTypes().));
            Schema contents = schema.getField("contents").schema();

            GenericArray <GenericRecord> people = new GenericData.Array<GenericRecord>(3, contents);

            Schema contentSchema = contents.getElementType();

With cricket_007 help, I am able to move forward a bit, after last line, should I have to create another array record as it has another array inside. or, should be able to set the record, this statement gives me little inside view of the record format but could not make it up .

System.out.println(contents.getElementType().getFields());

It's an array.

Try making a GenericData.Array

How to populate data for contents

There's an add method for arrays

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM