Goal: Move a lot of files from AWS S3 to ADLS Gen2 using Azure Synapse as fast as possible using parameterized regex expression for filename pattern using Synapse Notebook.
What I tried so far:
Question: Do I have to use the existing linked service using the credentials.getFullConnectionString(LinkedService) API ? In short, my question is, How do I configure connectivity to S3 from within Synapse Notebook?
Answering my question here. AzCopy worked.Below is the link which helped me finish the task. The steps are as follows.
https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10
https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-s3
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.