如何在 Apache Flink 中 flatMap 到数据库？

Question

I am using Apache Flink trying to get JSON records from Kafka to InfluxDB, splitting them from one JSON record into multiple InfluxDB points in the process.

I found the flatMap transform and it feels like it fits the purpose. Core code looks like this:

DataStream<InfluxDBPoint> dataStream = stream.flatMap(new FlatMapFunction<JsonConsumerRecord, InfluxDBPoint>() {
    @Override
    public void flatMap(JsonConsumerRecord record, Collector<InfluxDBPoint> out) throws Exception {
        Iterator<Entry<String, JsonNode>> iterator = //...

        while (iterator.hasNext()) {
            // extract point from input
            InfluxDBPoint point = //...

            out.collect(point);
        }
    }
});

For some reason, I only get one of those collected points streamed into the database.

Even when I print out all mapped entries, it seems to work just fine: dataStream.print() yields:

org.apache.flink.streaming.connectors.influxdb.InfluxDBPoint@144fd091
org.apache.flink.streaming.connectors.influxdb.InfluxDBPoint@57256d1
org.apache.flink.streaming.connectors.influxdb.InfluxDBPoint@28c38504
org.apache.flink.streaming.connectors.influxdb.InfluxDBPoint@2d3a66b3

Am I misunderstanding flatMap or might there be some bug in the Influx connector?

Answer 1

问题实际上与 Influx 中的一个系列（由其标签集和测量定义，如下所示）每次只能有一个点有关，因此即使我的字段不同，最后一点用相同的时间覆盖了所有以前的点价值。