使用数据块集群执行 azure 存储上存在的 python 代码

Question

We are having code structure like

In pkg folder we have main.py and execute.py imports (from pkg import main.hello)

execute.py

from mnt.cdr.code.pkg import main

main.hello()

main.py

def hello():
    print("hello")

this code is present on azure storage and that blob container is mounted to /mnt/cdr/code.. But we are getting "mnt Module not found".

How can I run this code from blob container on azure databricks cluster?

Answer 1

The Module Not Found Error usually occurs when Python cannot locate an error. The most common cause of this error is not installing modules required or importing a module incorrectly.

please check if all modules are imported.If all are correct please restart the cluster and check

The issue can be due to version of azure library which may conflict with the already present libraries.Check by downgrading or upgrading Azure library. Also When you try to access an already created mount point or create a new mount point, it fails with the error.

To unmount a mount point, use the following command: dbutils.fs.unmount("/mnt/")

check Failure when mounting Blob storage - Azure Databricks | Microsoft Docs & bash - Azure Databricks cluster init script - install python wheel - Stack Overflow

References:

1. azure-sdk-for-python-storage-blob-upload-download
2. Libraries | Databricks
3. util-linux