Is the flink MapState's TTL for the whole MapState instance or for each element in the MapState

Question

I have a MapState in flink. And I set the ttl to 10 minutes. Is the ttl for the whole MapState instance or for each of the elements?

    val ttlConfig = StateTtlConfig
      .newBuilder(Time.minutes(10))
      .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
      .setStateVisibility(StateTtlConfig.StateVisibility.NeverReturnExpired)
      .build

      val myMapState: MapState[String, Upload] = ....

      myMapState.put("a", "x")

      // 5 minutes later
      myMapState.put("b", "y")

      // Again 6 minutes later, is the myMapState still available? 
      // I assume myMapState is still available and it still has key "b" for 4 minutes. Is that correct?

Answer 1

Hi your assumption is correct.

If a TTL is configured and a state value has expired, the stored value will be cleaned up on a best effort basis.

So the value might still be stored even if the TTL has passed, the state visibility configuration will control if the value will be returned or not.

• StateTtlConfig.StateVisibility.NeverReturnExpired - expired value is never returned

• StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp - returned if still available

More info about state TTL

Answer 2

Yes, flink supports ttl per entry.

State Time-To-Live (TTL) A time-to-live (TTL) can be assigned to the keyed state of any type. If a TTL is? configured and a state value has expired, the stored value will be cleaned up on a best effort basis which is discussed in more detail below.

All state collection types support per-entry TTLs. This means that list elements and map entries expire independently.

https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl