How to configure flink SQL client to submit job with checkpointing enabled?

Question

What is an example of a flinksql-client config file with checkpointing enabled?

When I submit streaming job through flink sql client, the checkpoint is not enabled.

Answer 1

The checkpointing configuration can not be set in flink sql client config file, but it can be set in the cluster configuration file (flink-conf.yaml).

At a minimum you should configure execution.checkpointing.interval , state.backend , and state.checkpoints.dir .

Something like this, for example:

execution.checkpointing.interval: 10000
state.backend: filesystem
state.checkpoints.dir: file:///tmp/flink-checkpoints-directory
state.savepoints.dir: file:///tmp/flink-savepoints-directory

See the configuration docs for checkpointing , fault tolerance , and state backends for more details.

Answer 2

As far as I see in the docs you should be able to do it from the sql client by setting the properties like this:

SET 'state.checkpoints.dir' = 'hdfs:///bar/foo/';
SET 'execution.checkpointing.mode' = 'EXACTLY_ONCE';
SET 'execution.checkpointing.interval' = '30min';
SET 'execution.checkpointing.min-pause' = '20min';
SET 'execution.checkpointing.max-concurrent-checkpoints' = '1';
SET 'execution.checkpointing.prefer-checkpoint-for-recovery' = 'true';