简体   繁体   English

如何使用flink计数窗口

[英]how to use flink count window

when i use countwindow before windowfunction, after running the code i got the error like State migration is currently not supported , detail msg is当我在 windowfunction 之前使用 countwindow 时,在运行代码后,我收到了错误,例如State migration is currently not supported ,详细信息为

org.apache.flink.util.FlinkRuntimeException: org.apache.flink.util.StateMigrationException: State migration is currently not supported.
    at org.apache.flink.runtime.state.heap.HeapKeyedStateBackend.create(HeapKeyedStateBackend.java:216)
    at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.createTimerPriorityQueue(InternalTimeServiceManager.java:121)
    at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.registerOrGetTimerService(InternalTimeServiceManager.java:106)
    at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.getInternalTimerService(InternalTimeServiceManager.java:87)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator.getInternalTimerService(AbstractStreamOperator.java:741)
    at org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.open(WindowOperator.java:225)
    at org.apache.flink.streaming.runtime.operators.windowing.EvictingWindowOperator.open(EvictingWindowOperator.java:430)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.StateMigrationException: State migration is currently not supported.
    at org.apache.flink.util.StateMigrationException.notSupported(StateMigrationException.java:42)
    ... 11 more

my generator code is我的生成器代码是

SingleOutputStreamOperator singleOutputStream = iotDataSource.flatMap(new String2MetricFlatMapFunctionWithOutTimeAlignment())
    .filter(new WaterMarkFilterFunction(Time.seconds(WATERMARK_SIZE)))
    .filter(new DataCleanFilterFunction())
    .map(new RelateParkMapFunction())
    .assignTimestampsAndWatermarks(new CommonBoundedOutOfOrdernessTimestampExtractor(Time.seconds(MAX_OUT_OF_ORDERNESS)))
    .keyBy((KeySelector<Metric, String>) metric -> metric.getPark())
    .countWindow(10,1)
    .apply(new RealTimeAlarmWindowApplyFunction(CACHE_SIZE))
    .map(new ResultMapFunction())
    .disableChaining();

maybe something wrong in RealTimeAlarmWindowApplyFunction ,也许RealTimeAlarmWindowApplyFunction

public class RealTimeAlarmWindowApplyFunction extends RichWindowFunction<Metric, ResultAction, String, GlobalWindow> {

    private Logger logger = LoggerFactory.getLogger(RealTimeAlarmWindowApplyFunction.class);

    private int cacheDataSize;

    public RealTimeAlarmWindowApplyFunction(int cacheDataSize) {
        this.cacheDataSize = cacheDataSize;
    }

    @Override
    public void apply(String key, GlobalWindow window, Iterable<Metric> input, Collector<ResultAction> out) throws Exception {
        //...
        // process code with no error 
        //...
    }
}

Now this question is fixed by cancled savepoit option when running flink job:现在这个问题在运行 flink 作业时被取消的 savepoit 选项解决了:

$FLINK_HOME/bin/flink run -ynm jobname -yjm 1024 -ytm 4096 -yn 1 -ys 4 -qu root.users.root -flink_job_jar /root/flink_jar/test/flinkJobTest.jar -args 16

ps: i run this job allways with savepoit option like this ps:我总是用这样的savepoit选项运行这个工作

$FLINK_HOME/bin/flink run -ynm jobname -yjm 1024 -ytm 4096 -yn 1 -ys 4 -qu root.users.root -flink_job_jar /root/flink_jar/test/flinkJobTest.jar -savepoint_prefix hdfs://nameservice1/flink-checkpoint -args 16

but after i change window type, savepoint is not allowed to restore但是在我更改窗口类型后,不允许恢复保存点

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM