Unable to refer python libraries from Nifi ExecuteScript processor

Question

I have been trying to run a python script in NiFi's ExecuteScript processor. Though the catch here is that I don't have server file location access and all the python libraries are installed at "/data/jython", "/data/jython/Lib/site-packages/" and "data/nltk"

Below is the import section of my python script:

import json, traceback, pycountry, requests, geocoder, re, sys, nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk.corpus import stopwords
from java.nio.charset import StandardCharsets
from org.apache.commons.io import IOUtils
from org.apache.nifi.processor.io import StreamCallback
from org.python.core.util import StringUtil

I have added path reference to the packages/libraries:

Heres the screenshot of the error message:

Is there something I am missing? I have referred to another answer here , but couldn't figure out whats wrong with my code.

Answer 1

As the other answers state, Apache NiFi's ExecuteScript processor uses Jython , not Python . There is a limitation on the Jython library that it cannot handle native modules (modules that end in .so or are compiled C code, etc.). It is very likely that the pycountry module contains some native module. You can try a work-around proposed by Matt Burgess on the NiFi Developers Mailing List here .

Answer 2

ExecuteScript processor uses its own Jython Engine to execute your python scripts. As the libraries which you are importing are not available in NIFI inbuild Jython Engine its throwing error.

SOLUTION:

If python is already installed on our machine with all those libraries (the same machine where your NIFI is installed) you can use that python engine to execute your script. you can execute your python code using ExecuteProcess processor. see the configuration of ExecuteProcess .