[英]kafka-connect-elasticsearch How to route multiple topics to same elasticsearch index in same connector?
Trying to create elasticsearch sink connector with following config, the creation is successful but when a message is produced on "my.topic.one", ES sink connector fails while trying to create an index with name "my.topic.one": "Could not create index 'my.topic.one'" (User that I am using to connect to ES does not have create index permission intentionally).尝试使用以下配置创建 elasticsearch 接收器连接器,创建成功,但是当在“my.topic.one”上生成消息时,ES 接收器连接器在尝试创建名称为“my.topic.one”的索引时失败:“无法创建索引'my.topic.one'”(我用来连接到 ES 的用户没有故意创建索引权限)。 Why is it trying to create a new index and how to get the connector to index to previously created "elasticsearch_index_name"?为什么要尝试创建新索引以及如何让连接器索引到以前创建的“elasticsearch_index_name”?
{
"type.name": "_doc",
"tasks.max": "1",
"connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
"connection.url": "http://elasticsearch-service:9200",
"behavior.on.null.values": "delete",
"key.ignore": "false",
"write.method": "upsert",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter.schemas.enable": "false",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter.schemas.enable": "false",
"topics": "my.topic.one,my.topic.two",
"transforms": "renameTopic",
"transforms.renameTopic.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.renameTopic.regex": ".*",
"transforms.renameTopic.replacement": "elasticsearch_index_name"
}
UPDATE: ES sink connector throws error even if I use just one topic in "topics" attribute and same topic name in "renameTopic.regex" like below, rest all attributes same.更新:即使我在“主题”属性中仅使用一个主题并且在“renameTopic.regex”中使用相同的主题名称,ES sink 连接器也会引发错误,如下所示,rest 所有属性都相同。
"topics": "my.topic.one",
"transforms.renameTopic.regex": "my.topic.one"
Adding following property to ES sink connector config, solved the issue at hand:将以下属性添加到 ES sink 连接器配置,解决了手头的问题:
"auto.create.indices.at.start": "false"
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.