Receive streaming data from dataset with custom receiver [Spark Streaming]

Question

I am new with Spark and it's librabries!

I have a case where I need to send a dataset to store() of custom receiver, then streaming data from this custom receiver.

I am getting a dataset in this way inside custom receiver class:

Dataset<Row> jdbcDF = spark.read()
.format("jdbc")
.option("url", "jdbc:mysql:dbserver")
.option("dbtable", "schema.tablename")
.option("user", "username")
.option("password", "password")
.load();

Now I want to use this dataset as streaming data, so I have to store this dataset in store() of Receiver class:

store(jdbcDF);

Is it possible to streaming in this way?

ps not using Structured Streaming Programming

Thank you in advance!

Answer 1

spark does not support streaming of a jdbc database table. but it does support as a receiver. Rather use a kafka connect to send your data from a DB table as a streaming service and read it via spark structured straming.