将 DataBrick 连接到 Azure Blob 存储

Question

I'm trying to mount an Azure Blob Storage Container to a DataBricks instance and while the mount does work, it appears not to be using the Storage Container.

Here is the code I'm using to create the mount...

storageAccountName = 'x'
blobContainerName = 'files'
storageAccountAccessKey = 'x'

dbutils.fs.mount(
  source = f'wasbs://{blobContainerName}@{storageAccountName}.blob.core.windows.net',
  mount_point = '/mnt/files/',
  extra_configs = {'fs.azure.account.key.' + storageAccountName + '.blob.core.windows.net': storageAccountAccessKey}
)

The mount seems to work sucessfully.

I then download a file into the mounted location...

%sh

wget https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2019-financial-year-provisional/Download-data/annual-enterprise-survey-2019-financial-year-provisional-csv.csv -P /mnt/files/

I then list the files within the mounted location

%sh

ls /mnt/samplefiles/

This does show the downloaded file (annual-enterprise-survey-2019-financial-year-provisional-csv.csv), but doesn't show the file already in the Storage Account, nor does the downloaded file every appear in the Storage Account.

What am I missing here?

Answer 1

I needed to add /dbfs/ to the mount location.

For example...

ls /dbfs/mnt/samplefiles