"Can not parse attribute value as date" in Nifi UpdateAttribute processor

Question

 <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <template encoding-version="1.3"> <description></description> <groupId>de9048c5-0171-1000-4355-5efd185aa6ad</groupId> <name>Update_Attribute_Date</name> <snippet> <processors> <id>61a62c99-d8fc-3810-0000-000000000000</id> <parentGroupId>b055aedb-ad9e-374d-0000-000000000000</parentGroupId> <position> <x>0.0</x> <y>0.0</y> </position> <bundle> <artifact>nifi-update-attribute-nar</artifact> <group>org.apache.nifi</group> <version>1.11.4</version> </bundle> <config> <bulletinLevel>WARN</bulletinLevel> <comments></comments> <concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount> <descriptors> <entry> <key>Delete Attributes Expression</key> <value> <name>Delete Attributes Expression</name> </value> </entry> <entry> <key>Store State</key> <value> <name>Store State</name> </value> </entry> <entry> <key>Stateful Variables Initial Value</key> <value> <name>Stateful Variables Initial Value</name> </value> </entry> <entry> <key>canonical-value-lookup-cache-size</key> <value> <name>canonical-value-lookup-cache-size</name> </value> </entry> <entry> <key>created_date</key> <value> <name>created_date</name> </value> </entry> <entry> <key>modified_date</key> <value> <name>modified_date</name> </value> </entry> <entry> <key>schema.name</key> <value> <name>schema.name</name> </value> </entry> <entry> <key>statement.type</key> <value> <name>statement.type</name> </value> </entry> </descriptors> <executionNode>ALL</executionNode> <lossTolerant>false</lossTolerant> <penaltyDuration>30 sec</penaltyDuration> <properties> <entry> <key>Delete Attributes Expression</key> </entry> <entry> <key>Store State</key> <value>Do not store state</value> </entry> <entry> <key>Stateful Variables Initial Value</key> </entry> <entry> <key>canonical-value-lookup-cache-size</key> <value>100</value> </entry> <entry> <key>created_date</key> <value>${created_date:toDate("EEE MMM dd HH:mm:ss z yyyy"):toNumber():format("yyyy-MM-dd HH:mm:ss.SSS")}</value> </entry> <entry> <key>modified_date</key> <value>${modified_date:toDate("EEE MMM dd HH:mm:ss z yyyy"):toNumber():format("yyyy-MM-dd HH:mm:ss.SSS")}</value> </entry> <entry> <key>schema.name</key> <value>village</value> </entry> <entry> <key>statement.type</key> <value>${cdc.event.type}</value> </entry> </properties> <runDurationMillis>0</runDurationMillis> <schedulingPeriod>0 sec</schedulingPeriod> <schedulingStrategy>TIMER_DRIVEN</schedulingStrategy> <yieldDuration>1 sec</yieldDuration> </config> <executionNodeRestricted>false</executionNodeRestricted> <name>Schema=village &amp; Statement Type</name> <relationships> <autoTerminate>false</autoTerminate> <name>success</name> </relationships> <state>RUNNING</state> <style/> <type>org.apache.nifi.processors.attributes.UpdateAttribute</type> </processors> </snippet> <timestamp>06/13/2020 22:39:36 IST</timestamp> </template>

I am trying to insert records from source to destination db using the CaptureChangeMySQL processor. Date value from source table "2020-03-31 23:08:47" is getting changed to a format "Tue Mar 31 23:08:47 IST 2020" when processed in "CaptureChangeMySQL". The change value could not be loaded in the destination table since the data type is DateTime and getting an error "truncation due to length". So I am trying to convert the input value to correct format "yyyy-MM-dd HH:mm:ss" in the "UpdateAttribute" processor. Getting error as in the attachment. Please suggest how this issue could be fixed.

Answer 1

@Ganesh Kumar the expression language you are looking for is as follows:

${testValue:toDate("EEE MMM dd HH:mm:ss z yyyy"):toNumber():format("yyyy-MM-dd HH:mm:ss.SSS")}

In my NiFi test, I set testValue = "Tue Mar 31 23:08:47 IST 2020" in updateAttribute 1, then direct to another updateAttribute 2. I did notice that my output is 3 hours off, but suspect that is just my local timezone difference.

The flowfile Output is:

test1
2020-03-31 21:08:47.000

testValue
Tue Mar 31 23:08:47 IST 2020

Edited 6/17/2020 After Comments Above:

Access your json array in EvaluateJsonPath for each element as follows:

$[0].created_date

Then use the ${created_date} attribute with the Expression Language we tested above in updateAttribute as follows:

${created_date:toDate("EEE MMM dd HH:mm:ss z yyyy"):toNumber():format("yyyy-MM-dd HH:mm:ss.SSS")}

You can find a template demo at (right side flow): https://github.com/steven-dfheinz/NiFi-Templates/blob/master/NiFI_EvaluateJsonPath_Demo.xml