简体   繁体   中英

Dstream Spark Streaming

Hello I wrote the following code

val receivedStream: DStream[PubsubSparkMessage] =
      PubsubSource.createStream(ssc, pubsubConfig, pubsubSubscriber)
    @volatile var receivedMessages: List[PubsubSparkMessage] = List() //scalastyle:ignore
    receivedStream.foreachRDD { rdd =>
      if (!rdd.isEmpty()) {
        receivedMessages = List(rdd.collect)
        receivedMessages
      }
    }

I am getting an error because type mismatch; My question is how to modify List[Array[PubsubSparkMessage]] to List[PubsubSparkMessage]

Something like:

val list : List[Array[PubsubSparkMessage]] = List()

val result = list.flatMap(x => x)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM