Data Factory v1将某些标头屏蔽为http标头中的凭据

Question

I have a Data Factory (v1) which downloads some files from an HTTP server.

Within the dataset pointing to the file location on this server we add an API key as an additional header to the HTTP request. We don't want this key to be visible from the portal similar to how Linked Services mask credentials after having been deployed.

The following Json files define the source linked service, the source dataset and the copy activity.

• HTTP_source_linkedservice.json

{ "name": "HTTPSourceLinkedService", "properties": { "hubName": "this_is_a_hubname", "type": "Http", "typeProperties": { "url": "https://website.com", "authenticationType": "Anonymous" } } }

• HTTP_source_dataset

{ "name": "HTTPSourceDataset", "properties": { "published": false, "type": "Http", "linkedServiceName": "HTTPSourceLinkedService", "typeProperties": { "relativeUrl": "/main_file_to_download", "additionalHeaders": "X-api-key: API_KEY_HERE\\n" }, "availability": { "frequency": "Day", "interval": 1 }, "external": true, "policy": {} } }

• Copy Activity

{ "type": "Copy", "typeProperties": { "source": { "type": "HttpSource" }, "sink": { "type": "BlobSink", "writeBatchSize": 0, "writeBatchTimeout": "00:00:00" } }, "inputs": [ { "name": "HTTPSourceDataset" } ], "outputs": [ { "name": "HTTPSinkDataset" } ], "scheduler": { "frequency": "Day", "interval": 1 }, "name": "CopyFileFromServer" }

I know we could use a Custom Activity to make the request itself and fetch the API key from a keyvault but I really want to use the standard Copy Activity.

Is there a way to achieve this ?

Answer 1

Unfortunately, I think this is not possible. Header field are defined as string. And in v1, there is even no secure string which is introduced in v2 to indicate a field is credentials. But I think this can't be achieved in v2 either. As the model type is fixed.