I want to debug my notebook thus I need to print out the streaming-data in notebook console mode. I have two questions: 1- Is it possible to do:
df.writeStream.format("console").start().awaitTermination()
2- If yes, where can I see the output?
This is after 10 minutes... No error or result
Thanks!
I figured out what the problem in my case was. I had to add the ".outputMode("append")" in my method. Here is how it looks:
def writeStreamData(dataFrame: DataFrame): Unit = {
/**
* write the given dataframe into a file or console
:params: dataframe
*/
dataFrame.writeStream
.format("console")
.outputMode("append")
.start()
.awaitTermination()
}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.