Error Databricks Scala Application with Azure Table Storage

Question

first of all thank you for your time for the next question:)

I am trying to connect Databricks Scala Application with Azure Table Storage, however I am getting the following error:

Azure Table Scala APP

Error:

NoSuchMethodError: reactor.netty.http.client.HttpClient.resolver(Lio.netty/resolver/AddressResolverGroup;)Lreactor.netty/transport/ClientTransport;

at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:94) at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:18) at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62) at com.azure.data.tables.BuilderHelper.buildPipeline(BuilderHelper.java:122) at com.azure.data.tables.TableServiceClientBuilder.buildAsyncClient(TableServiceClientBuilder.java:161) at com.88 33723520088.data.tables.TableServiceClientBuilder.buildClient(TableServiceClientBuilder.java:93)

I attach the code:

val clientCredential: ClientSecretCredential  = new ClientSecretCredentialBuilder()
.tenantId(tenantID)
.clientId(client_san_Id)
.clientSecret(client_san_Secret)
.build()


val tableService = new TableServiceClientBuilder()
.endpoint("https://<Resource-Table>.table.core.windows.net")
.credential(clientCredential)
.buildClient()

Thank you very much for your time!

Answer 1

First you need to mount Storage on Azure databricks.

Then use the code below to mount Table Storage.

dbutils.fs.mount(
  source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/<directory-name>",
  mountPoint = "/mnt/<mount-name>",
  extraConfigs = Map("<conf-key>" -> dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")))

Access table storage using below code:

// scala
val df = spark.read.text("/mnt/<mount-name>/...")
val df = spark.read.text("dbfs:/<mount-name>/...")

You can refer this notebook

Also refer this article by Gauri Mahajan