使用 NiFi 仅使用新的 ElasticSearch 文档

Question

I have a use case where I have to periodically fetch new records using NiFi (only records not fetched before) for a particular index and pipe it to other storage doing some ETL.

I'm not sure how to handle fetching only new records. Did anyone handle use cases like that?

Answer 1

can you be more specific about fetching only new records ? when you say, only records not fetched before , does that mean with a specific timeline? what records are being termed as new records? Is there any time-specific requirement?