简体繁体 English

如何停止Spark Streaming

[英]How to Stop Spark Streaming

原文 2015-07-29 12:01:03 1 1 scala/ twitter/ apache-spark/ streaming/ connector

I am working on twitter connector. 我正在使用twitter连接器。 I want to stop streaming after one minute. 我想在一分钟后停止播放。 Can anyone give idea ho to do this? 任何人都可以提出想法吗？ I tried ssc.awaitTermination(60000). 我试过ssc.awaitTermination（60000）。 but it didn't work. 但它不起作用。

1 个解决方案

Note that awaitTermination with a timeout is deprecated: 请注意，不推荐使用带有超时的awaitTermination ：

https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala#L633 https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala#L633

However with either awaitTermination or awaitTerminationOrTimeout you should follow it with ssc.stop to stop the context once the timeout is reached. 但是，无论是awaitTermination还是awaitTerminationOrTimeout都应该使用ssc.stop跟随它，以便在达到超时后停止上下文。

如何在Spark流媒体上创建停止条件？ - How to create a stop condition on Spark streaming?

停止spark流式上下文kafkaDirectStream - stop spark Streaming context kafkaDirectStream

Spark流上下文停止 - Spark streaming context hangs on stop

如何限制Spark Streaming？ - How to throttle Spark Streaming?

如何在火花流中对数据进行排序 - How to sort data in spark streaming

如何检查许多火花流的来源 - How to checkpoint many source of spark streaming

如何编写Spark Streaming程序的单元测试？ - How to write unit tests for Spark Streaming programs?

如何在Spark Streaming作业中查找数据框的大小 - How to find the size of the dataframe in spark streaming jobs

reduceByKeyAndWindow如何在Spark流中工作？ - How does reduceByKeyAndWindow works in Spark streaming?

Spark：如何使用RowEncoder创建流数据集？ - Spark: How to create streaming Dataset with RowEncoder?

暂无

暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何在Spark流媒体上创建停止条件？ - How to create a stop condition on Spark streaming? 停止spark流式上下文kafkaDirectStream - stop spark Streaming context kafkaDirectStream Spark流上下文停止 - Spark streaming context hangs on stop 如何限制Spark Streaming？ - How to throttle Spark Streaming? 如何在火花流中对数据进行排序 - How to sort data in spark streaming 如何检查许多火花流的来源 - How to checkpoint many source of spark streaming 如何编写Spark Streaming程序的单元测试？ - How to write unit tests for Spark Streaming programs? 如何在Spark Streaming作业中查找数据框的大小 - How to find the size of the dataframe in spark streaming jobs reduceByKeyAndWindow如何在Spark流中工作？ - How does reduceByKeyAndWindow works in Spark streaming? Spark：如何使用RowEncoder创建流数据集？ - Spark: How to create streaming Dataset with RowEncoder?

相关标签

粤ICP备18138465号 © 2020-2024 STACKOOM.COM