有没有办法使用apache nifi中的处理器从json字段提取值并将该值替换为另一个json文件？

Question

So I have created a workflow in apache nifi that pulls csv attachments from gmail and converts them to json. What I am stuck on is extracting 3 values (clientip,Country,user-agent) from the json I currently have and replacing those values within another json that I have that will be used to run alerts on another program. I am not sure as to what processors would be used to achieve this. Any tips would be greatly appreciated.

I have tried playing around with extract attributes and JoltTransformJson but I can not get either of them to work as intended..

First json that I get from converting csv file:

{
  "clientip" : "116.255.157.126",
  "Country" : "China",
  "host" : "teachinglaw-prod.uis.georgetown.edu",
  "useragent" : "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)",
  "uri" : "//Config_Shell.php"
}

Second json I wrote that needs to have ("data","message","data") values updated with first json:

    {
"title": "cat7-SQL Injection",
"description": "alert description",
"type": "Internal ",
"source": "Splunk ",
"sourceRef": "Splunk alert ",
"severity": 2,
"tlp": 2,
"artifacts": [{
"dataType": "ip",
"data": "176.121.14.180",
"message": "Belize",
"tags": ["SQL Injection"]
},
{
"dataType": "user - agent",
"data": "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.53 Safari/525.19",
"tags": ["SQL Injection"]
}
],
"caseTemplate": "SQL Injection"
}

I need to find a processor(s) that will give me this result after values have been merged/substituted :

    {
"title": "cat7-SQL Injection",
"description": "alert description",
"type": "Internal ",
"source": "Splunk ",
"sourceRef": "Splunk alert ",
"severity": 2,
"tlp": 2,
"artifacts": [{
"dataType": "ip",
"data": "116.255.157.126",
"message": "China",
"tags": ["SQL Injection"]
},
{
"dataType": "user - agent",
"data": "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)",
"tags": ["SQL Injection"]
}
],
"caseTemplate": "SQL Injection"
}

Answer 1

I would suggest using the EvaluateJSONPath processor to extract the desired JSON values to flowfile attributes, and then route to ReplaceText and use Expression Language to replace template tokens with the attribute values. For example, given this "input JSON":

{
  "clientip" : "116.255.157.126",
  "Country" : "China",
  "host" : "teachinglaw-prod.uis.georgetown.edu",
  "useragent" : "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)",
  "uri" : "//Config_Shell.php"
}

Your EvaluateJsonPath processor should have the following configuration (any property not listed is kept as the default, and the last 3 are "Dynamic Properties" added with the " + " button on the top right of the table):

• Destination : flowfile-attribute
• ip : $.clientip
• message : $.Country
• user_agent : $.useragent

The next processor is a ReplaceText processor with the following configuration:

• Search Value : (?s)(^.*$)
• Replacement Value : {...JSON template below...}

The JSON template is the following:

{
    "title": "cat7-SQL Injection",
    "description": "alert description",
    "type": "Internal ",
    "source": "Splunk ",
    "sourceRef": "Splunk alert ",
    "severity": 2,
    "tlp": 2,
    "artifacts": [{
            "dataType": "ip",
            "data": "<template_ip>",
            "message": "<template_message>",
            "tags": ["SQL Injection"]
        },
        {
            "dataType": "user - agent",
            "data": "<template_user_agent>",
            "tags": ["SQL Injection"]
        }
    ],
    "caseTemplate": "SQL Injection"
}

Each value of the form <template_some_value> will get matched and the some_value piece will get extracted as the capture group referenced by $2 .

Finally, another ReplaceText is used to inject the attribute value into each template token location. The configuration is:

• Search Value : <(template_(\\w+))>
• Replacement Value : ${${'$2'}}

The final output will look like: