[英]Reading data from Azure Blob Storage into Azure Databricks using /mnt/
I've successfully mounted my blob storage to Databricks, and can see the defined mount point when running dbutils.fs.ls("/mnt/")
. 我已经成功地将我的blob存储装载到Databricks,并且在运行
dbutils.fs.ls("/mnt/")
时可以看到定义的挂载点。 This has size=0
- it's not clear if this is expected or not. 这个
size=0
- 不清楚这是否是预期的。
When I try and run dbutils.fs.ls("/mnt/<mount-name>")
, I get this error: java.io.FileNotFoundException: / is not found
当我尝试运行
dbutils.fs.ls("/mnt/<mount-name>")
,我收到此错误: java.io.FileNotFoundException: / is not found
When I try and write a simple file to my mounted blob with dbutils.fs.put("/mnt/<mount-name>/1.txt", "Hello, World!", True)
, I get the following error (shortened for readability): 当我尝试使用
dbutils.fs.put("/mnt/<mount-name>/1.txt", "Hello, World!", True)
将一个简单文件写入我的挂载blob时, dbutils.fs.put("/mnt/<mount-name>/1.txt", "Hello, World!", True)
以下错误(缩短了可读性):
ExecutionError: An error occurred while calling z:com.databricks.backend.daemon.dbutils.FSUtils.put. : shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.
...
Caused by: com.microsoft.azure.storage.StorageException: The specified resource does not exist.
All the data is in the root of the Blob container, so I have not defined any folder structures in the dbutils.fs.mount
code. 所有数据都在Blob容器的根目录中,因此我没有在
dbutils.fs.mount
代码中定义任何文件夹结构。
thinking emoji 思维表情符号
The solution here is making sure you are using the 'correct' part of your Shared Access Signature (SAS). 此处的解决方案是确保您使用的是共享访问签名(SAS)的“正确”部分。 When the SAS is generated, you'll find there are lots of different parts of it that you can use - it's likely sent to you as one long connection string, eg:
生成SAS时,您会发现它可以使用很多不同的部分 - 它可能作为一个长连接字符串发送给您,例如:
BlobEndpoint=https://<storage-account>.blob.core.windows.net/;QueueEndpoint=https://<storage-account>.queue.core.windows.net/;FileEndpoint=https://<storage-account>.file.core.windows.net/;TableEndpoint=https://<storage-account>.table.core.windows.net/;SharedAccessSignature=sv=<date>&ss=nwrt&srt=sco&sp=rsdgrtp&se=<datetime>&st=<datetime>&spr=https&sig=<long-string>
When you define your mount point, use the value of the SharedAccessSignature
key, eg: 定义安装点时,请使用
SharedAccessSignature
键的值,例如:
sv=<date>&ss=nwrt&srt=sco&sp=rsdgrtp&se=<datetime>&st=<datetime>&spr=https&sig=<long-string>
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.