How do I combine table.include.list and column.include.list in Debezium connector configuration if I need to snapshot part of one table and full data from another table?
Example connector configuration:
"table.include.list":"schema.table1, schema.table2",
"column.include.list":"schema.table1.col1, schema.table1.col2"
With this I get all columns of table1 in kafka.
Only managed how to do it using different connectors.
I want to get all columns from table2 and 2 columns from table1 using one connector, is it possible?
This should work:
"table.include.list":"schema\.table1,schema\.table2",
"column.include.list":"schema\.table1\.{col1|col2},schema\.table2\..*"
From Debezium documentation for column.include.list configuration attribute:
An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. Fully-qualified names for columns are of the form databaseName.tableName.columnName.
So you have to configure 2 things:
If you want to include all columns from table, then configuration of column names in the column_include_list attribute is not necessary.
For example:
{
...
"table.include.list": "schema\\.table1,schema\\.table2",
"column.include.list": "schema\\.table1\\.(col1|col2)"
...
}
Column exclusion is configured in the same way, but using the column_exclude_list attribute.
See details here (ie for MySQL connector, other connectors are similar): https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-table-include-list
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.