[英]Updating a Debezium MySQL connector with table whitelist option
I'm using the Debezium (0.7.5) MySQL connector and I'm trying to understand what is the best approach if I want to update this configuration with the option table.whitelist<\/code> .
我正在使用 Debezium (0.7.5) MySQL 连接器,如果我想使用选项
table.whitelist<\/code>更新此配置,我试图了解什么是最佳方法。
Let's say I create a connector, something like this:假设我创建了一个连接器,如下所示:
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://debezium-host/connectors/ -d '
{
"name": "MyConnector",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"connect.timeout.ms": "60000",
"tasks.max": "1",
"database.hostname": "myhost",
"database.port": "3306",
"database.user": "***",
"database.password": "***",
"database.server.id": "3227197",
"database.server.name": "MyServer",
"database.whitelist": "myDb",
"table.whitelist": "myDb.table1,myDb.table2",
"database.history.kafka.bootstrap.servers": "kb0:9092,kb1:9092,kb2:9092",
"database.history.kafka.topic": "MyConnectorHistoryTopic",
"max.batch.size": "1024",
"snapshot.mode": "initial",
"decimal.handling.mode": "double"
}
}'
Changes to the whitelist/blacklist config are not yet supported at this point.目前尚不支持对白名单/黑名单配置的更改。 This is currently being worked on (see DBZ-175 ), and we hope to have preview support for this in one of the next releases.
目前正在对此进行处理(请参阅DBZ-175 ),我们希望在下一个版本中提供对此的预览支持。 There's a pending PR for this, which needs a bit more work, though.
不过,这有一个悬而未决的 PR ,需要做更多的工作。
Until this has been implemented, your best option is to set up a new instance of the connector which only captures the additional tables you're interested in. This comes at the price of running two connectors (which both will maintain a binlog reader session), but it does the trick as long as you don't need to change your filter config too often.在实现之前,您最好的选择是设置一个新的连接器实例,它只捕获您感兴趣的附加表。这是以运行两个连接器为代价的(这两个连接器都将维护一个 binlog 读取器会话) ,但只要您不需要经常更改过滤器配置,它就可以解决问题。
The latest version of Debezium Server, you can add the following config Debezium Server 最新版本,可以添加如下配置
debezium.snapshot.new.tables=parallel
In case If you are using Debezium, you can try this config value如果您使用的是 Debezium,您可以试试这个配置值
snapshot.new.tables=parallel
Note: Debeziyum Server is the one that supports Kinesis, Google Pub sub, and Apache Pulsar.注意:Debeziyum Server 是一种支持 Kinesis、Google Pub sub 和 Apache Pulsar 的服务器。 I am using that and its configuration is a bit different.
我正在使用它,它的配置有点不同。 I had to prepend "debezium" before each item
我必须在每个项目之前加上“debezium”
Once this configuration is added, any addition to tables.whitelist, For these additional tables Debezium will create snapshots.添加此配置后,任何添加到tables.whitelist 中,Debezium 都会为这些额外的表创建快照。
I cannot point you to the documentation but I went through their code in GitHub and also I tried it practically which worked for me.我无法将您指向文档,但我在 GitHub 中浏览了他们的代码,并且我实际上尝试了它,这对我有用。 Here is the link to the MySqlConnector code
这是 MySqlConnector 代码的链接
https://github.com/debezium/debezium/blob/master/debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlConnectorConfig.java https://github.com/debezium/debezium/blob/master/debezium-connector-mysql/src/main/java/io/debezium/connector/mysql/MySqlConnectorConfig.java
There search for Field.create("snapshot.new.tables")那里搜索 Field.create("snapshot.new.tables")
Personally, I feel like Debezium has a lot of things but documentation is scattered.就我个人而言,我觉得 Debezium 有很多东西,但文档很分散。
i am have the same problem and solve with a signal table to debezium.我有同样的问题,并用 debezium 的信号表解决。 Its work that way, you have to create a table to send to debezium commands in your datatable.
它以这种方式工作,您必须创建一个表以发送到数据表中的 debezium 命令。
CREATE TABLE public.debezium_signal (id VARCHAR(42) PRIMARY KEY, type VARCHAR(32) NULL, data VARCHAR(2048) NULL);
and set in your configuration do debzium a tag "signal.data.collection": "public.debezium_signal"
并在你的配置中设置做 debzium 一个标签
"signal.data.collection": "public.debezium_signal"
after that you can send commands with insert in that table:之后,您可以在该表中使用 insert 发送命令:
INSERT INTO debezium_signal (id, type, data)
VALUES(gen_random_uuid(),'execute-snapshot','{"data-collections": "myDb.table3"]}');
in my case i have to add de table signal in table.include.list and the columns in column.include.list as well.在我的情况下,我必须在 table.include.list 和 column.include.list 中的列中添加 de table 信号。
https://debezium.io/documentation/reference/stable/configuration/signalling.html https://debezium.io/documentation/reference/stable/configuration/signalling.html
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.