Azure Data Factory V2 logical OR

Question

Is it possible to create a logical OR for Azure Data Factory V2 pipeline activities? For example, we have two copy activities and on fails of any of its the Data Factory should call an API.

Answer 1

Yes, we can.

You could add a If Condition to create the logical pipeline. If the two copy active output status contains 'failed', then call the API.

As we know, each copy active has the output has the bellow details:

"executionDetails": [
        {
            "source": {
                "type": "AzureSqlDatabase",
                "region": "East US"
            },
            "sink": {
                "type": "AzureSqlDatabase",
                "region": "East US"
            },
            "status": "Succeeded",
            "start": "2020-07-08T05:37:33.9079553Z",
            "duration": 4,
            "usedDataIntegrationUnits": 4,
            "usedParallelCopies": 1,
            "profile": {
                "queue": {
                    "status": "Completed",
                    "duration": 2
                },
                "transfer": {
                    "status": "Completed",
                    "duration": 1,
                    "details": {
                        "readingFromSource": {
                            "type": "AzureSqlDatabase",
                            "workingDuration": 0,
                            "timeToFirstByte": 0
                        },
                        "writingToSink": {
                            "type": "AzureSqlDatabase",
                            "workingDuration": 0
                        }
                    }
                }
            },

We can set the executionDetails:{"status": "Succeeded"} to the if-condition active:

Expression: No matter which actives failed, all the API:

@or(equals(activity('Copy data1').output.executionDetails.status,'Failed'),equals(activity('Copy data2').output.executionDetails.status,'Failed') )

If condition true, add the true active to call the API:

If all the copy actives are "succeeded"(condition false) and we don't want to do anything, we don't need to add the false active.

Hope this helps.

Answer 2

Not that I know about. When an activity has multiple conditions, they are treated as an AND condition, meaning that both copy activities have to fail to execute the web activity.

One common pattern to do this is create a pipeline that encapsulates the logic to report errors, and call it from each activity's failurem, capturing error output to a pipeline parameter.

Hope this helped!