How to load a Processor into Apache NiFi while it is running?

Question

I it possible to load a new Processor into Apache NiFi while it is running Furthermore, is it possible to update an already loaded Processor in a running Apache NiFi instance?

I am coming from an Erlang background, where the BeamVM can actually facilitate hot module replacements all together with state updates. Its a complicated topic, but possible.

It would be great to at least be able to load new code into a running NiFi instance, without having to stop the cluster when applying new Processors.

---

A possible workaround I have found (although not preferred) is to use a scripting Processor that can set/update code at runtime via its properties.

Answer 1

Historically it has required a full restart of NiFi to load new NARs, but on the apache master branch there is a new feature to watch a directory and auto-load new NARs, so you can expect this to be available in a future release, possibly 1.9.0 if the community decides on that.

https://issues.apache.org/jira/browse/NIFI-5673

We don't plan to re-load existing NARs as it can be problematic to unload classes in the JVM, and also with versioned NARs I would argue that when you deploy a NAR you should be deploying a new version, and not trying to overwrite a version of an already deployed NAR.

Hope that helps.