还有job.coordinator.system的替代选择

Question

I want to use samza, but case is our kafka topic creation is limited (topic creation should be reviewed and should has concrete porpose). So, is there any other choice for "job.coordinator.system"? And I need the usage intro. Thanks a lot!

Answer 1

As of Samza 0.12.0, Samza auto-creates 3 categories of streams:

1. Coordinator stream to durably store job model information, eg the host affinity mapping.
2. Checkpoint stream to store checkpoints for consumption of input streams
3. Changelog streams for any stores with changelog enabled

It is very common to use Kafka log-compacted topics for each of these, but theoretically any System implementation would work. You even can define your own.

Some details on Samza's stream abstractions can be found here: http://samza.apache.org/learn/documentation/0.12/container/streams.html