简体   繁体   English

kafka如何处理对同一代理上多个分区的写入,而不会因磁盘搜寻而减慢速度

[英]How does kafka handles writes to multiple partitions on same broker without slowing down as disk seek happens

If 2 topics containing the leaders on same broker are receiving frequent writes, the disk head will seek from one topic partition disk location to another topics's disk location. 如果包含同一代理上的领导者的2个主题频繁收到写入,则磁盘头将从一个主题分区磁盘位置搜索到另一个主题的磁盘位置。 Since its no longer a sequential write will it not slow down the writing to the topics? 由于它不再是顺序写入,是否会减慢主题的写入速度?

the simple answer is that kafka lies :-) 简单的答案是,卡夫卡说谎:-)

a kafka broker acknowledges a write once data has been written to page cache . 一旦将数据写入页面缓存, kafka代理便会确认写入。 data is then sync'ed to disk in the background. 数据然后在后台同步到磁盘。

the advantage is that the operating system page cache is free to reorder the writes to be more sequential (although in general youre right - having lots of partitions on the same disk results in less and less linearity in writes). 优点是操作系统页面高速缓存可以自由地将写操作重新排序,使其更具顺序性(尽管通常来说,在同一磁盘上具有很多分区会导致写线性度越来越低)。

the obvious disadvantage is that you can lose acknowledged data on things like power loss 明显的缺点是您可能会丢失诸如断电之类的已确认数据

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM