Kafka Mongo Sink 连接器，如何将 SMT 时间戳转换器用于 JSON 数组中归档的日期

Question

"transforms": "TimestampConverter",

"transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.format": "yyyy-MM-dd",
"transforms.TimestampConverter.target.type": "string",
"transforms.TimestampConverter.field" : "DateAtlevel2"

When this date filed is at parent level of JSON the SMT works fine, its not working when Date filed inside a JSOn array.

How to apply the SMT to convert a filed inside JSON array into a Date?

I am using Kafka Mongo Sink connector.

Answer 1

I assume that you are using the org.apache.kafka.connect.json.JsonConverter.java which is a schemaless converter. In this case, you would only be able to retrieve the level 1 fields.

Why? During the SMT phase, each record is stripped down to a plain Map<String, Object>, where String is the field name and the Object is its value. What the TimestampConverter does is to simply do a get on that map with the field name that you've passed as a config ( map.get("DateAtlevel2") ) which it can't be found.

To get access to level 2+ fields you would need some other type of input data that supports schemas ( AVRO, JSONSchema, or Protobuff ).