How Flink Kafka Producer set Semantic in Scala

Question

I want to set Flink kafka producer semantic.EXACTLY_ONCE, I'm using scala code as following

alertEnrichStream.map(_.toJsonStr)
  .addSink(
    new FlinkKafkaProducer011(
      kafkaBrokers,
      kafkaOutputTopic,
      new SimpleStringSchema))

How to edit the code to set Semantic?

Answer 1

Use the constructor that can define the semantic. Like this one:

FlinkKafkaProducer011<Integer> kafkaProducer = new FlinkKafkaProducer011<>(
    topic,
    new SimpleStringSchema,
    properties,
    Semantic.EXACTLY_ONCE);

Answer 2

I've come up with this instructor usage and it works pretty fine:

val producerProps = new Properties
producerProps.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers)
producerProps.setProperty(ProducerConfig.RETRIES_CONFIG, "2147483647")
producerProps.setProperty(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "1")
producerProps.setProperty(ProducerConfig.ACKS_CONFIG, "all")
producerProps.setProperty(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true")

new FlinkKafkaProducer011[String](
  topic,
  new KeyedSerializationSchemaWrapper[String](new SimpleStringSchema),
  producerProps,
  Optional.of(new FlinkFixedPartitioner[String]),
  FlinkKafkaProducer011.Semantic.EXACTLY_ONCE,
  10
)