Azure ADF v2-如何向要部署到Azure Data Lake的文件名添加/附加时间戳

Question

I am deploying a file to azure datalake but want to append datetime stamp to filename . I created a dataset and created a parameter with value as utcnow(). But file is not getting deployed with timestamp appended.

{

"name": "tst",

"properties": {

    "linkedServiceName": {

        "referenceName": "LS_ADLS_AzureDataLakeStoreLinkedService",

        "type": "LinkedServiceReference"

    },

    "parameters": {

        "customVariable": {

            "type": "String",

            "defaultValue": "utcnow()"

        }

    },

    "type": "AzureDataLakeStoreFile",

    "typeProperties": {

        "format": {

            "type": "TextFormat",

            "columnDelimiter": "|",

            "rowDelimiter": "",

            "nullValue": "",

            "treatEmptyAsNull": true,

            "skipLineCount": 0,

            "firstRowAsHeader": true

        },

        "fileName": "abc-{customVariable}",

        "folderPath": "clusters/diageo-eun-analytics-nonprod-hdi-hd-nampolaris-dev01/hive/warehouse/Spirit_Deal/tst",

        "partitionedBy": [

            {

                "name": "customVariable",

                "value": {

                    "type": "DateTime",

                    "date": "SliceStart",

                    "format": "yyyyMMddHH"

                }

            }

        ]

    }

},

"type": "Microsoft.DataFactory/factories/datasets"

}

But filename is now coming with timestamp appended ie abc-{customVariable}.Please tell how to add in ADV v2

Answer 1

You definitely can parameterize file names and pathes. Phps. you try @CONCAT. The following example works, showing how to build folder pathes and filenames dynamically:

{
    "name": "TgtADLSSQL",
    "properties": {
        "linkedServiceName": {
            "referenceName": "Destination-DataLakeStore-gqf",
            "type": "LinkedServiceReference"
        },
        "folder": {
            "name": "Target"
        },
        "type": "AzureDataLakeStoreFile",
        "typeProperties": {
            "format": {
                "type": "ParquetFormat"
            },
            "fileName": {
                "value": "@CONCAT('Ingest_Date=', formatDateTime(trigger().startTime, 'yyyy-MM-dd/'),'Incremental-', pipeline().RunId, '.snappy.parquet')",
                "type": "Expression"
            },
            "folderPath": {
                "value": "/data/raw/corporate/sql/@{item().TABLE_NAME}",
                "type": "Expression"
             }
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

See also Dynamic folders in ADF V2 .