Apache NiFi 1.10 : How to execute a python script as a processor?

Question

In NiFi <= v1.9 I used the following NiFi setup:

GetTwitter generates a JSON for every tweet gathered and attached to it there used to be a ExecuteStreamCommand that has the following parameters:

This would allow me to get the flowfile as input, and I could analyse the content and do something (in this specific case I would put the content on Elasticsearch) - You could reach the same result if you do something like this cat flowfile.json | python <scriptname>.py cat flowfile.json | python <scriptname>.py

But now in >= v1.10 it seems I cannot do the same. it gives me some error about parameters (it seems it wants to use the flowfile as parameter on calling the command/script instead, as it used to be, pass the flowfile as input. It gives me the following error:

2019-12-16 14:28:44,953 ERROR [Timer-Driven Process Thread-2] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=ef1d15fb-016e-1000-3050-35fba1733f97] Transferring flow file StandardFlowFileRecord[uuid=c485d8aa-58e9-428c-84b1-b4acbd6275a7,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1576505352573-22, container=default, section=22], offset=663999, length=-1],offset=0,name=242802f9-5ffb-4a2c-90a5-27736459dfd0.json,size=0] to nonzero status. Executable command bash ended in an error: bash: line 1: $'{delete:status:id:1005156317567188992}\r': command not found
2019-12-16 14:28:44,962 ERROR [Timer-Driven Process Thread-2] o.a.n.p.standard.ExecuteStreamCommand ExecuteStreamCommand[id=ef1d15fb-016e-1000-3050-35fba1733f97] Transferring flow file StandardFlowFileRecord[uuid=5b617840-ba8e-47b3-9a27-06e8ec0f88bd,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1576505352573-22, container=default, section=22], offset=663999, length=-1],offset=0,name=eb9e236a-1ccb-4243-aa1c-e5de43746f80.json,size=0] to nonzero status. Executable command bash ended in an error: bash: line 1: $'{delete:status:id:1139832600422277120}\r': command not found

It seems it takes the flowfile JSON as parameter instead of input . Any ideas?

Answer 1

well, This is not really a problem with NiFi per se: NiFi was installed with docker so I kept using the docker host as it was the correct machine.

Embarassing.. :)