[英]how to use flink count window
當我在 windowfunction 之前使用 countwindow 時,在運行代碼后,我收到了錯誤,例如State migration is currently not supported
,詳細信息為
org.apache.flink.util.FlinkRuntimeException: org.apache.flink.util.StateMigrationException: State migration is currently not supported.
at org.apache.flink.runtime.state.heap.HeapKeyedStateBackend.create(HeapKeyedStateBackend.java:216)
at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.createTimerPriorityQueue(InternalTimeServiceManager.java:121)
at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.registerOrGetTimerService(InternalTimeServiceManager.java:106)
at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.getInternalTimerService(InternalTimeServiceManager.java:87)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.getInternalTimerService(AbstractStreamOperator.java:741)
at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.open(WindowOperator.java:225)
at org.apache.flink.streaming.runtime.operators.windowing.EvictingWindowOperator.open(EvictingWindowOperator.java:430)
at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.StateMigrationException: State migration is currently not supported.
at org.apache.flink.util.StateMigrationException.notSupported(StateMigrationException.java:42)
... 11 more
我的生成器代碼是
SingleOutputStreamOperator singleOutputStream = iotDataSource.flatMap(new String2MetricFlatMapFunctionWithOutTimeAlignment())
.filter(new WaterMarkFilterFunction(Time.seconds(WATERMARK_SIZE)))
.filter(new DataCleanFilterFunction())
.map(new RelateParkMapFunction())
.assignTimestampsAndWatermarks(new CommonBoundedOutOfOrdernessTimestampExtractor(Time.seconds(MAX_OUT_OF_ORDERNESS)))
.keyBy((KeySelector<Metric, String>) metric -> metric.getPark())
.countWindow(10,1)
.apply(new RealTimeAlarmWindowApplyFunction(CACHE_SIZE))
.map(new ResultMapFunction())
.disableChaining();
也許RealTimeAlarmWindowApplyFunction
,
public class RealTimeAlarmWindowApplyFunction extends RichWindowFunction<Metric, ResultAction, String, GlobalWindow> {
private Logger logger = LoggerFactory.getLogger(RealTimeAlarmWindowApplyFunction.class);
private int cacheDataSize;
public RealTimeAlarmWindowApplyFunction(int cacheDataSize) {
this.cacheDataSize = cacheDataSize;
}
@Override
public void apply(String key, GlobalWindow window, Iterable<Metric> input, Collector<ResultAction> out) throws Exception {
//...
// process code with no error
//...
}
}
現在這個問題在運行 flink 作業時被取消的 savepoit 選項解決了:
$FLINK_HOME/bin/flink run -ynm jobname -yjm 1024 -ytm 4096 -yn 1 -ys 4 -qu root.users.root -flink_job_jar /root/flink_jar/test/flinkJobTest.jar -args 16
ps:我總是用這樣的savepoit選項運行這個工作
$FLINK_HOME/bin/flink run -ynm jobname -yjm 1024 -ytm 4096 -yn 1 -ys 4 -qu root.users.root -flink_job_jar /root/flink_jar/test/flinkJobTest.jar -savepoint_prefix hdfs://nameservice1/flink-checkpoint -args 16
但是在我更改窗口類型后,不允許恢復保存點
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.