nifi + executescript使用python计算文件中的行

Question

Hi i need to get the numbers of line in a csv file exclusive the first line which are headers I need to store the number of lines in a attribute and parse the flowfile untouched to the next processor

I was thinking of using extracttext but i don't think that a regular expression can do this.

So next step would be a executeScript processor. I was think of a python script with following template

flowFile = session.get() 
if (flowFile != None):
# All processing code starts at this indent
attrMap = ['numberOflines': '1', 'myAttr2': Integer.toString(2)]
flowFile = session.get()
if(!flowFile) return
#Do something to get numbers of lines in the flow file
i =0;
    for line in flowfile
        i+=1

flowFile = session.putAttribute(flowFile, 'attribute_numberOfLines', i)
if errorOccurred:
    session.transfer(flowFile, REL_FAILURE)
else:
    session.transfer(flowFile, REL_SUCCESS)

implicit return at the end

This will not run

Answer 1

Try a SplitText processor with the Line Split Count set to some number higher than the largest number of possible lines in your files (such as 1 million). You can also set the Header Line Count to 1 if you want the total number of lines minus the header. You'll get the same flow file(s) out, but with an attribute text.line.count that will contain the number of lines.