MiNiFi GetFile 处理器无法获取大文件

Question

I'm running apache MiNiFi c++ , The flow starts with a GetFile processor.

The input directory includes some large files, and when I run MiNiFi the files above ~1.5 GB fail and do not get queued.

The log file states:

[org::apache::nifi::minifi::processors::GetFile] [Warning] failed to stat large_file_path_here

The other smaller files are queued as expected.

Does anyone have a clue what can be wrong? Why can't the processor manage the larger files?

Thanks in advance.

Answer 1

What you found seems like a bug that is present even in the current MiNiFi implementation even up to today. The issue is that file sizes you mentioned, a narrowing exception happens here when trying to determine the length of the file to be written into the content repository.

We will try to fix this issue asap.