Migrating Data from a SQL Server Encrypted Table to SQL Azure using Azure Data Factory Copy data

Question

I am having difficulties with Azure Data Factory migrating data from an encrypted table. I was wondering what I have missed as I can't find a way to attach a certificate in the pipeline to extract the data.

I have a Link Service connection that Connects Successfully, but I am guessing it is possibly here that the certificate needs to be added for the decrypting on the data.

In 'Copy Data' the Source dataset, the Linked service connects to the database fine. But the Table, preview data Fails.

I get a 22301 Error. The Detail is as follows:

A database operation failed with the following error: Failed to decrypt column... Failed to decrypt a column encryption key using key store provider: 'MSSQL_CERTIFICATE_STORE'. The last 10 bytes of the encrypted column encryption key are: '...'. Certificate with thumbprint '...' not found in certificate store 'My' in certificate location 'CurrentUser'. Verify the certificate path in the column master key definition in the database is correct, and the certificate has been imported correctly into the certificate location/store.

Parameter name: masterKeyPath

Failed to decrypt column... Failed to decrypt a column encryption key using key store provider: 'MSSQL_CERTIFICATE_STORE'. The last 10 bytes of the encrypted column encryption key are: '...'.

Certificate with thumbprint '...' not found in certificate store 'My' in certificate location 'CurrentUser'. Verify the certificate path in the column master key definition in the database is correct, and the certificate has been imported correctly into the certificate location/store.

Parameter name: masterKeyPath, SqlErrorNumber=0,Class=11,State=0,

Certificate with thumbprint '...' not found in certificate store 'My' in certificate location 'CurrentUser'. Verify the certificate path in the column master key definition in the database is correct, and the certificate has been imported correctly into the certificate location/store.

Parameter name: masterKeyPath Activity ID: ...

I have no idea what this means, is there someone who can explain what the problem is and how I can rectify this I would be extremely grateful.

What I am trying to achieve, with very little success is to migrate the data from the encrypted data table in one database unencrypted to another. I want to use an Azure Data factory pipeline and make redundant SSIS.

Many thanks

Answer 1

To transfer Encrypted data from data factory you have to use the Azure key vault :

• Open the SQL Server Management Studio and connect with the server .
• Expand Database >> security >> Always Encrypted keys >> New Column master key.

Give the master key a name, Choose the Azure Key Vault as your key storage location, login into your account, choose your Azure subscription , choose the Azure key vault , generate the key , and then click "ok."

• Expand Database >> security >> Always Encrypted keys >> New Column Encryption key.

Name your column encryption key , choose the master key and click on ok.

• Encrypt columns

To Encrypt a column or columns, right-click on the Table , click on "Encrypt Columns " and select single or multiple columns as per your requirement to encrypt.

• Access policies

To add an access policy, open your Azure portal , navigate to the Azure key vault , select the settings tab's access policies link, and then click + Add access policy . Select the principal , which is your Azure data factory name , before choosing the important rights. Click on Add after selecting Secret Permissions, followed by Certificate Permissions .

• linked service configuration Create linked service , Name linked service, select Azure subscription , select the server name , select the database name , select the authentication type, provide the user name and password , select always encrypted select keystore authentication type as System Assigned Managed Identity and then click ok create .

Create pipeline select source and sink and run the pipeline

Reference: Using Always Encrypted