How to use delta trigger in flink?

Question

I want to use the deltatrigger in apache flink (flink 1.3) but I have some trouble with this code :

.trigger(DeltaTrigger.of(100, new DeltaFunction[uniqStruct] {
    override def getDelta(oldFp: uniqStruct, newFp: uniqStruct): Double = newFp.time - oldFp.time
  }, TypeInformation[uniqStruct]))

And I have this error:

error: object org.apache.flink.api.common.typeinfo.TypeInformation is not a value [ERROR] }, TypeInformation[uniqStruct]))

I don't understand why DeltaTrigger need TypeSerializer[T] and I don't know what to do to remove this error.

Thanks a lot everyone.

Answer 1

I would read into this a bit https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/types_serialization.html sounds like you can create a serializer using typeInfo.createSerializer(config) on your type info. Note what you're passing in currently is a type itself and NOT the type info which is why you're getting the error you are.

You would need to do something more like

val uniqStructTypeInfo: TypeInformation[uniqStruct] = createTypeInformation[uniqStruct]
val uniqStrictTypeSerializer = typeInfo.createSerializer(config)

To quote the page above regarding the config param you need to pass to create serializer

The config parameter is of type ExecutionConfig and holds the information about the program's registered custom serializers. Where ever possibly, try to pass the programs proper ExecutionConfig. You can usually obtain it from DataStream or DataSet via calling getExecutionConfig(). Inside functions (like MapFunction), you can get it by making the function a Rich Function and calling getRuntimeContext().getExecutionConfig().

Answer 2

DeltaTrigger needs a TypeSerializer because it uses Flink's managed state mechanism to store each element for later comparison with the next one (it just keeps one element, the last one, which is updated as new elements arrive).

You will find an example (in Java) here .

But if all you need is a window that triggers every 100msec, then it'll be easier to just use a TimeWindow , such as

input
  .keyBy(<key selector>)
  .timeWindow(Time.milliseconds(100)))
  .apply(<window function>)

Updated:

To have hour-long windows that trigger every 100msec, you could use sliding windows. However, you would have 10 * 60 * 60 windows, and every event would be placed into each of these 36000 windows. So that's not a great idea.

If you use a GlobalWindow with a DeltaTrigger , then the window will be triggered only when events are more than 100msec apart, which isn't what you've said you want.

I suggest you look at ProcessFunction . It should be straightforward to get what you want that way.