Apache Nifi 中的更多处理器是否会导致吞吐量降低？

Question

In Apache Nifi, there are connections between each processors, which acts like queue of FlowFiles, and Nifi by default persists data content of FlowFile on disk. Does it mean each of such connection persists FlowFiles on disk? If that were true, each time of delivery of FlowFiles from one processor to another would mean one disk read and write, thus more processors would lead to more disk reads and writes, which in turn would lower the entire throughput. Is my understanding correct? and what is the best practice to avoid it, writing all things in one processor? Thanks.

Answer 1

NiFi internals are a bit different. Attribute values persist in memory and FlowFile content persists on disk. So if the processor is doing an operation on FlowFile attributes such as UpdateAttribute then no need to access the content but the operation is happening on content (Data Enrichment) such as ValidateRecord then disk IO will be involved. If you observe any processor, you can see Read/Write stats, this tells you the amount of IO that happened. Refer to this, Anatomy of a Processor , for more details.

If you have a custom logic that needs to modify both attributes and content then you can implement both the operations in one custom processor!