Stream data into Azure Databricks using Event Hubs

Question

I want to send messages from a Twitter application to an Azure event hub. However, I am getting the an error that says instead of java.util.concurrent.ExecutorService use java.util.concurrent.ScheduledExecutorService .

I do not know how to create the EventHubClient.create now. Please help.

I am referring to code from the link

https://docs.microsoft.com/en-us/azure/azure-databricks/databricks-stream-from-eventhubs

This is error I am getting:

notebook:15: error: type mismatch; found : java.util.concurrent.ExecutorService required: java.util.concurrent.ScheduledExecutorService

 val pool = `Executors.newFixedThreadPool(1)` val eventHubClient = EventHubClient.create(connStr.toString(), pool) 

Here is my code,

import java.util._
import scala.collection.JavaConverters._
import com.microsoft.azure.eventhubs._
import java.util.concurrent.{Executors, ExecutorService}

val pool: ExecutorService = Executors.newFixedThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)

Answer 1

Create a library in your Databricks workspace using the Maven coordinates com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.2 or higher version. Attach the created library to your cluster and re-attach the notebook to your cluster after it.

import java.util._
import scala.collection.JavaConverters._
import com.microsoft.azure.eventhubs._
import java.util.concurrent._
//Set up Connection to Azure Event Hubs
val namespaceName = "namespaceName_of_event_hub"
val eventHubName = "Name_of_event_hub"
val sasKeyName = "your_Sas_key_name"
val sasKey = "xxxxxxxxxxxxxxxxxxxxxxx"
val connStr = new ConnectionStringBuilder()
            .setNamespaceName(namespaceName)
            .setEventHubName(eventHubName)
            .setSasKeyName(sasKeyName)
            .setSasKey(sasKey)
val pool = Executors.newFixedThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)