带有 Azure 表存储的错误 Databricks Scala 应用程序

Question

首先感谢您抽空回答下一个问题：）

我正在尝试将 Databricks Scala 应用程序与 Azure 表存储连接起来，但是出现以下错误：

Azure 表 Scala APP

错误：

NoSuchMethodError: reactor.netty.http.client.HttpClient.resolver(Lio.netty/resolver/AddressResolverGroup;)Lreactor.netty/transport/ClientTransport;

at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:94) at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:18) at com.azure.core.implementation.http.HttpClientProviders.createInstance (HttpClientProviders.java:58) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) at com.azure.core.http.HttpPipelineBuilder. build(HttpPipelineBuilder.java:62) at com.azure.data.tables.BuilderHelper.buildPipeline(BuilderHelper.java:122) at com.azure.data.tables.TableServiceClientBuilder.buildAsyncClient(TableServiceClientBuilder.java:161) at com.88 33723520088.data.tables.TableServiceClientBuilder.buildClient(TableServiceClientBuilder.java:93)

我附上代码：

val clientCredential: ClientSecretCredential  = new ClientSecretCredentialBuilder()
.tenantId(tenantID)
.clientId(client_san_Id)
.clientSecret(client_san_Secret)
.build()


val tableService = new TableServiceClientBuilder()
.endpoint("https://<Resource-Table>.table.core.windows.net")
.credential(clientCredential)
.buildClient()

非常感谢您的宝贵时间！

Answer 1

首先，您需要在 Azure 数据块上安装存储。

然后使用下面的代码挂载Table Storage。

dbutils.fs.mount(
  source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/<directory-name>",
  mountPoint = "/mnt/<mount-name>",
  extraConfigs = Map("<conf-key>" -> dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")))

使用以下代码访问表存储：

// scala
val df = spark.read.text("/mnt/<mount-name>/...")
val df = spark.read.text("dbfs:/<mount-name>/...")

你可以参考这个笔记本

另请参阅 Gauri Mahajan 的这篇文章