How can I read PostgreSQL Table partitions with AWS Glue Crawler?

Question

My Crawler is not able to read the RDS partitioned table properly when partitions are created in Postgresql 11.

Example of partition DDL is

Table:

CREATE TABLE book (
    ID int8 NULL,
    effectivetodate date NULL,
) PARTITION BY RANGE(effectivetodate);

Partitions:

CREATE TABLE book_historical PARTITION OF portfolio for VALUES FROM ('1000-12-31') TO ('2019-12-31');
CREATE TABLE book_current PARTITION OF portfolio DEFAULT;

The observation is that two tables:

• book_historical
• book_current

are created by Crawler but the main table

• book

is not there.

Answer 1

INstead of using range just pass the column name for partitioning and crawler will dynamically partition based on your column

CREATE TABLE book (
    ID int8 NULL,
    effectivetodate date NULL,
) PARTITION BY effectivetodate;