Access composite data in Postgres via Apache Drill

Question

I am fairly new to Apache Drill. I need to query my PostgreSQL database using Apache Drill. I went through the documentation and could configure a storage plugin for my Postgres database.

{

  "type": "jdbc",

  "driver": "org.postgresql.Driver",

  "url": "jdbc:postgresql://localhost/mydb",

  "username": <username>,

  "password": <password>,

  "enabled": true

}

I can successfully query tables with primary datatypes but unable to query the ones that have composite datatype. I have tried querying it using various ways, example .

This is the exception i get:

Caused by: java.lang.NullPointerException: null
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:201)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructure(RelStructuredTypeFlattener.java:225)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:205)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:184)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:435)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.drill.exec.planner.sql.SqlConverter.toRel(SqlConverter.java:270)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRel(DefaultSqlHandler.java:638)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:196)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:165)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:131)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79)
~[drill-java-exec-1.11.0.jar:1.11.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1050)
[drill-java-exec-1.11.0.jar:1.11.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:280)
[drill-java-exec-1.11.0.jar:1.11.0]

Is querying composite datatype in drill not feasible at all? Please let me know if am missing something.

Thanks

Answer 1

Apache Drill v1.11 documentation for data types: https://drill.apache.org/docs/supported-data-types/

In this release of Drill, you cannot reference a composite type by name in a query, but Drill supports array values coming from data sources.

EDIT

I suggest trying the ways provided by the Apache Drill community in the documentation. This is a bit from the section Composite Types :

For example, you can use the index syntax to query data and get the value of an array element:

a[1]

You can refer to the value for a key in a map using dot notation:

tmk

The section “ Query Complex Data ” shows how to use composite types to access nested arrays. " Handling Different Data Types " includes examples of JSON maps and arrays. Drill provides functions for handling array and map types:

• " KVGEN "
• " FLATTEN "