AWS Glue. How to create a compound key for Job bookmarks?

Question

I have a JDBC source (PostgreSQL) with a table, which I want to fetch by Glue.

My table has columns:

id          (bigint)
name        (string)
updated_at  (timestamp)

I've set up the table in the Glue data catalog with a crawler, set up a job and enabled Job bookmarks.

And when I run the job, it automatically defines new rows by new ids.

But I want to use the compound key -> [ id + updated_at ].

It will alow me to detect all updates in the source table.

How can I do it?

AWS docs say that this feature is available ( https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html ):

For JDBC sources, the following rules apply:
   * For each table, AWS Glue uses one or more columns as bookmark keys to determine new and processed data. The bookmark keys combine to form a single compound key.
   * You can specify the columns to use as bookmark keys. If you don't specify bookmark keys, AWS Glue by default uses the primary key as the bookmark key, provided that it is sequentially increasing or decreasing (with no gaps).

Should I define the table manually (without crawlers)?

Thanks !

Answer 1

datasource0 = glueContext.create_dynamic_frame.from_catalog(
    database = "hr", table_name = "emp",
    transformation_ctx = "datasource0",
    additional_options = {
        "jobBookmarkKeys": ["empno"],
        "jobBookmarkKeysSortOrder": "asc"
    }
)