简体   繁体   English

Kafka Streams 中流流连接的默认 WindowBytesStoreSupplier 是什么?

[英]What is the default WindowBytesStoreSupplier for stream-stream join in Kafka Streams?

The new API has a signature of:新的 API 具有以下特征:

join(KStream<K,VO> otherStream, ValueJoiner<? super V,? super VO,? extends VR> joiner, JoinWindows windows, StreamJoined<K,V,VO> streamJoined)

If I only set store name using StreamJoined<K,V,VO> streamJoined parameter, what would be the default configuration of WindowBytesStoreSupplier in terms of retentionPeriod , windowSize and retainDuplicates ?如果我只使用StreamJoined<K,V,VO> streamJoined参数设置商店名称,那么WindowBytesStoreSupplierretentionPeriodwindowSizeretainDuplicates方面的默认配置是什么? It's not clear from documentation.从文档中不清楚。 Is this an in memory state store?这是在 memory state 商店吗? Or default one configured by Kafka Streams?还是 Kafka Streams 配置的默认配置?

If you only set the store name with StreamJoined then:如果您仅使用StreamJoined设置商店名称,则:

  1. The windowSize comes from what you provide the JoinWindows configuration object. windowSize来自您提供的JoinWindows配置 object。 Even if you use a custom WindowBytesStoreSupplier , Kafka Streams validates that the supplier window settings match the settings of the provided JoinWindows object.即使您使用自定义WindowBytesStoreSupplier ,Kafka Streams 也会验证供应商 window 设置是否与提供的JoinWindows object 的设置相匹配。 So a JoinWindows.of(Duration.ofSeconds(30)) would have a windowSize equal to 30000 ms.所以JoinWindows.of(Duration.ofSeconds(30))windowSize等于30000毫秒。
  2. The retentionPeriod is the window size + the grace period. retentionPeriod期是 window 大小 + 宽限期。 The default grace period is 24 hours.默认宽限期为 24 小时。
  3. The retainDuplicates configuration is true . retainDuplicates配置为true But even when providing a custom StoreSupplier , the retainDuplicates field must be set to true .但即使在提供自定义StoreSupplier时, retainDuplicates字段也必须设置为true
  4. The store type is a persistent (RocksDB) store, configured by Kafka Streams, as described above.存储类型是持久化(RocksDB)存储,由 Kafka Streams 配置,如上所述。

Also, note that with StreamJoined , you can now provide your own StoreSupplier for both sides of the join, so it's possible to have in-memory stores.另外,请注意,使用StreamJoined ,您现在可以为连接的双方提供自己的StoreSupplier ,因此可以拥有内存存储。

HTH,高温下,

Bill账单

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM