How to specify converter for default value in Avro Union logical type fields?

Question

I have the following Avro schema:

{
   "namespace":"com.example",
   "type":"record",
   "name":"BuggyRecord",
   "fields":[
      {
         "name":"my_mandatory_date",
         "type":{
            "type":"long",
            "logicalType":"timestamp-millis"
         },
         "default":1502250227187
      },
      {
         "name":"my_optional_date",
         "type":[
            {
               "type":"long",
               "logicalType":"timestamp-millis"
            },
            "null"
         ],
         "default":1502250227187
      }
   ]
}

I use maven to generate the Java file. Avro config:

       <plugin>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro-maven-plugin</artifactId>
            <version>1.8.2</version>
            <executions>
                <execution>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>schema</goal>
                        <!--<goal>idl-protocol</goal>-->
                    </goals>
                    <configuration>
                        <sourceDirectory>${project.basedir}/src/main/avro</sourceDirectory>
                        <outputDirectory>${project.basedir}/src/main/java</outputDirectory>
                        <enableDecimalLogicalType>true</enableDecimalLogicalType>
                        <stringType>String</stringType>
                    </configuration>
                </execution>
            </executions>
        </plugin>

The generated class through mvn compile code fails on some basic code:

@Test
public void Foo(){
    BuggyRecord.Builder buggyRecordBuilder = BuggyRecord.newBuilder();
    buggyRecordBuilder.build();
}

Error code:

org.apache.avro.AvroRuntimeException: java.lang.ClassCastException: java.lang.Long cannot be cast to org.joda.time.DateTime

    at com.example.BuggyRecord$Builder.build(BuggyRecord.java:301)
    at BuggyRecordTest.Foo(BuggyRecordTest.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to org.joda.time.DateTime
    at com.example.BuggyRecord$Builder.build(BuggyRecord.java:298)
    ... 23 more

Which I think is due to the Converters not being generated properly:

  private static final org.apache.avro.Conversion<?>[] conversions =
      new org.apache.avro.Conversion<?>[] {
      TIMESTAMP_CONVERSION,
      null,  // <------ THIS ONE IS NOT SET PROPERLY
      null
  };

Is it me mis-using the code generator or is it a bug?

Answer 1

  {
     "name":"my_optional_date",
     "type":[ "null", {"type" : "long", "logicalType": "timestamp-millis"}], "default": null
  }

More of a workaround than a solution - put "null" type first and "default": null

Answer 2

We ran into a similar issue, with logicalType of date:

{
  "name": "date_field",
  "type": [
    "null",
    {
      "type": "int",
      "logicalType": "date"
    }
  ],
  "default": null
}

We needed to update to the latest version ( 0.17.0 ) of the gradle plugin gradle-avro-plugin , and the Apache Avro dependency ( 1.9.1 ).

Also in our gradle build file, we needed to add

avro {
  dateTimeLogicalType = "JODA"
}

as well as a dependency on joda-time

dependencies {
    compile "joda-time:joda-time:2.10.3"
}

The default dateTimeLogicalType used by gradle-avro-plugin is JSR310, but that still caused errors (even though we are using Java 8).

Answer 3

Try upgrading the version of avro to 1.9.X in avro dependency and avro-maven-plugin in pom.xml

    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.9.1</version>
    </dependency>

    <plugin>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro-maven-plugin</artifactId>
        <version>1.9.1</version>
        <executions>
            <execution>
                <phase>generate-sources</phase>
                <goals>
                    <goal>schema</goal>
                    <!--<goal>idl-protocol</goal>-->
                </goals>
                <configuration>
                    <sourceDirectory>${project.basedir}/src/main/avro</sourceDirectory>
                    <outputDirectory>${project.basedir}/src/main/java</outputDirectory>
                    <enableDecimalLogicalType>true</enableDecimalLogicalType>
                    <stringType>String</stringType>
                </configuration>
            </execution>
        </executions>
    </plugin>

Also make sure to delete previously generated class from AVRO schema and do mvn compile.