AWS DataPipeline：SellCommandPrecondition不会失败

Question

Using ShellCommandPrecondition for a SQLActivity with Datapipeline. In the python script, if I use sys.exit(0) (success signal) then the Precondition returns successfully and pipeline continues to execute. But if I use sys.exit(1) (error signal) then the Precondition does not return and stays in Running state indefinitely.

But with SellCommandActivity, if I use sys.exit(1) then it fails gracefully and following activities fail.

Why ShellCommandPrecondition does not exhibit same behavior as SellCommandActivity or is this is a bug in AWS Datapipeline and needs to rectified?

Answer 1

The three properties to look at are:

• maximumRetries
• retryDelay
• preconditionTimeout

If your script fails (with a non-zero return code) on the first attempt, the precondition will not immediately fail. Instead it will pause for retryDelay and execute again up to maximumRetries . The total amount of time the precondition will allow for all attempts is determined by preconditionTimeout .