简体   繁体   中英

FastAvro Schema Issues

I've almost finished the avro section but I am having one big challenge ahead of me which is the name and namespace. I've tried this and no matter what I've done, I've experienced errors, even when referencing documents like: docs.oracle.com/cd/E26161_02/html/GettingStartedGuide/avroschemas.html

My schema header is: https://i.ibb.co/TPH8shF/image.png

With the error being: https://i.ibb.co/BCcX6Jd/image.png

However, when I add schemas to the namespace (since it's in the schemas folder), this error comes up: https://i.ibb.co/Z62RwN2/image.png

Do any of you know what could be causing this error? Thanks!

Here's the text version, if you need it:

src_data_path = 'data/processed/openflights/routes.jsonl.gz'
parsed_schema = load_schema("routes.avsc")
avro_output_path = results_dir.joinpath('results/routes.avro')
src_data_path = 'routes.jsonl'
with open(src_data_path, 'r') as f:
    avro_reader = json_reader(f, parsed_schema)        
    for record in avro_reader:
        print(record)
{
  "type": "record",
  "name": "routes",
  "namespace": "schemas",
  "fields": [
    {
      "name": "airline",
      "type": {
        "type": "record",
        "name": "Airline",
        "fields": [
          {
            "name": "airline_id",
            "type": "int",
            "default": -1
          },
          {
            "name": "name",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "alias",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "iata",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "icao",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "callsign",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "country",
            "type": "string",
            "default": "NONE"
          },
          {
            "name": "active",
            "type": "boolean",
            "default": false
          }
        ]
      },
      "default": "NONE"
    },
    {
      "name": "src_airport",
      "type": [
        {
          "type": "record",
          "name": "Airport",
          "fields": [
            {
              "name": "airport_id",
              "type": "int",
              "default": -1
            },
            {
              "name": "name",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "city",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "iata",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "icao",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "latitude",
              "type": "double"
            },
            {
              "name": "longitude",
              "type": "double"
            },
            {
              "name": "timezone",
              "type": "double"
            },
            {
              "name": "dst",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "tz_id",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "type",
              "type": "string",
              "default": "NONE"
            },
            {
              "name": "source",
              "type": "string",
              "default": "NONE"
            }
          ]
        },
        "null"
      ],
      "default": "NONE"
    },
    {
      "name": "dst_airport",
      "type": [
        "record",
        "null"
      ],
      "fields": [
      {
       "name": "source",
       "type": "string",
       "default": "NONE"
      }
      ],
      "default": "NONE"
    },
    {
      "name": "codeshare",
      "type": "boolean",
      "default": false
    },
    {
      "name": "stops",
      "type": "int",
      "default": 0
    },
    {
      "name": "equipment",
      "type": {
        "type": "array",
        "items": "string"
      }
    }
  ]
}

In the definition for dst_airport you have the following:

    {
      "name": "dst_airport",
      "type": [
        "record",
        "null"
      ],
      "fields": [
      {
       "name": "source",
       "type": "string",
       "default": "NONE"
      }
      ],
      "default": "NONE"
    },

The type of ["record", "null"] doesn't make sense because you haven't defined a named type called "record" (and I'm not sure the specification even allows you to do that. Either way, I'm assuming it should be ["Airport", "null"] .

Also, the fields here doesn't really make sense. It can't be a union type with fields .

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM