如何检查许多火花流的来源

Question

I have many CSV spark.readStream in a different locations, I have to checkpoint all of them with scala, I specified a query for every stream but when I run the job, I got this message

java.lang.IllegalArgumentException: Cannot start query with name "query1" as a query with that name is already active

I solved my problem by creating a many streaming query like this :

val spark = SparkSession
    .builder
    .appName("test")
    .config("spark.local", "local[*]")
    .getOrCreate()
spark.sparkContext.setCheckpointDir(path_checkpoint)
val event1 = spark  
.readStream //  
.schema(schema_a)  
.option("header", "true")    
.option("sep", ",")    
.csv(path_a) 

val query = event1.writeStream
  .outputMode("append")
  .format("console")
  .start()
   spark.streams.awaitAnyTermination()

Answer 1

val spark = SparkSession
    .builder
    .appName("test")
    .config("spark.local", "local[*]")
    .getOrCreate()
spark.sparkContext.setCheckpointDir(path_checkpoint)
val event1 = spark  
.readStream //  
.schema(schema_a)  
.option("header", "true")    
.option("sep", ",")    
.csv(path_a) 

val query = event1.writeStream
  .outputMode("append")
  .format("console")
  .start()
   spark.streams.awaitAnyTermination()