如何减少Alpakka的Kafka日志文件大小

Question

I am doing data replication in alpakka using Consumer.commitableSource. But, the size of kafka log file is increases very quickly. The size reaches 5 gb in a day. As a solution of this problem, ı want to delete processed data immediately. I am using delete record method in AdminClient to delete offset. But when I look at the log file, data corresponding to that offset is not deleted.

Answer 1

When using commitableSource you need to acknowledge that the record has been successfully read and is ready to be cleaned by committing the offset. You can do that by calling commitJavadsl() . Take a look at the example in the documentation for more information.