Azure Data Factory V2 using yesterdays date when triggering a pipeline run

Question

I have an Azure Data Factory (v2) that I use to backup the contents of a database to a blob store nightly (2am UTC). However, I expect the name of the file to contain the day of the month ( dd ) that the backup was generated, but it's always the day before.

The file name is generated using an expression -

@{formatDateTime(pipeline().parameters.windowStart,'dd')}.json

So for example the run at 3am today should have been called 23.json , but it was actually called 22.json . 3am is the expected run time as I'm in the UK, which is currently on BST (UTC+1)

Looking at the parameters of the run, I can see that the windowStart is indeed a day out. For example, todays run which was triggered at 2am on the 23rd had 9/22/2020 2:00:00 AM .

Is anybody able to explain why Data Factory is behaving in this way, and hopefully how I can make it work as expected.

Here is the trigger as exported from the Data Factory.

{
    "name": "Trigger_Copy_Transactions",
    "properties": {
        "annotations": [],
        "runtimeState": "Started",
        "pipeline": {
            "pipelineReference": {
                "referenceName": "Copy_Transactions",
                "type": "PipelineReference"
            },
            "parameters": {
                "windowStart": "@trigger().outputs.windowStartTime"
            }
        },
        "type": "TumblingWindowTrigger",
        "typeProperties": {
            "frequency": "Hour",
            "interval": 24,
            "startTime": "2020-08-24T02:00:00Z",
            "delay": "00:00:00",
            "maxConcurrency": 50,
            "retryPolicy": {
                "intervalInSeconds": 30
            },
            "dependsOn": []
        }
    }
}

Answer 1

One thing you could try is to force the file to be generated in the same time zone that your IR is running in. For example we have a Self Hosted IR so when we would generate files it would not match EST times. In that case I did the following:

@concat('File_name',formatDateTime(
convertFromUtc(utcnow(),'Eastern Standard Time'),'yyyy-MM-dd'),'.txt')

Perhaps doing that would force the proper date?

Are you also using Auto-generated IR or Self-Hosted IR when running this job?