Azure Databricks mounting a blob storage

Question

I'm trying to map a blob storage from my storage account into my databricks cluster. I'm following this link and here apparently I just need to execute this code in my notebook:

Code python:

dbutils.fs.mount(
  source = "wasbs://<your-container-name>@<your-storage-account-name>.blob.core.windows.net",
  mount_point = "/mnt/<mount-name>",
  extra_configs = {"<conf-key>":dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")})

I'm stuck in the parameters : (scope = "<scope-name>", key = "<key-name>") . I know that in order to create an scope i can follow this link , later I suppose to navigate my clister throughout the Databricks CLI and catch the <scope-name> and <key-name> . However, when I check my cluster I just get the scope name and I can't find the key name anywhere or at lkeast I dont know how to get it.

Tried on Databricks CLI:

Where can I generate or find the <key-name>

Answer 1

My understanding is that the Key name is the "thing" you are trying to retrieve from the secret scope.

I guess you created a databricks backed secret scope, not an Azure key vault backed secret scope?

You should have then actually stored a secret in the secret scope. That secret is stored against a "key" (confusing terminology).

https://docs.azuredatabricks.net/user-guide/secrets/secrets.html#create-a-secret-in-a-databricks-backed-scope

The code you have is trying to retrieve that secret back out of the secret scope and assign it to the <conf-key> in that json.

Have a look at this example https://docs.azuredatabricks.net/user-guide/secrets/example-secret-workflow.html#use-the-secrets-in-a-notebook

val jdbcUsername = dbutils.secrets.get(scope = "jdbc", key = "username")

Prior to this you need to create a key vault and inside it is a secret name username that you have already populated. Then this code goes and retrieves it

So.... you need to actually know what you want to put against in the first place and go put that in the secret scope.

As an example you can make your code simpler but very insecure by simply hard coding that setting in your code

Answer 2

Please note that the blobEndpoint source has to be in format :

wasbs://container@storageaccount.blob.core.windows.net

Then you set your variables:

blobEndpoint = dbutils.secrets.get("blob-secret","blob-endpoint")

blobKey = dbutils.secrets.get("blob-secret","blobl-access-key")

    dbutils.fs.mount(
  source = blobEndpoint,
  mount_point = "/mnt/blobmountsecret",
  extra_configs = `{"fs.azure.account.key.<storageaccountname>.blob.core.windows.net":blobKey})`