Empty timestamp columns to 'NULL'

Question

Below is my sample JSON data in which I had given few date columns having timestamp data type as empty. Now while ingesting JSON to DB I want the columns having data type as timestamp which are empty ('') to be converted to 'NULL'.

I have used a Replacetext processor to search for '' after split and Replacement value as 'Null' but if there are any other columns which are empty are also converting to 'Null' (like age). I only want to replace the timestamp columns to 'Null'.

Any suggestions on this issue?

[
  {
    "name": "Tony",
    "age": 22,
    "regdate": "2022-07-01 02:15:15",
    "due_date": "",
    "start_date": ""
  },
  {
    "name": "Steve",
    "age": 21,
    "regdate": "",
    "due_date": "2022-03-01 05:22:15",
    "start_date": ""
  },
  {
    "name": "Peter",
    "age": 23,
    "regdate": "",
    "due_date": "",
    "start_date": "2021-08-06 02:20:15"
  }
]

I have used a Replacetext processor to search for '' after split and Replacement value as 'Null' but if there are any other columns which are empty are also converting to 'Null' (like age). I only want to replace the timestamp columns to 'Null'.

Answer 1

Adding a JoltTransformJSON processor which converts a JSON value to another formatted JSON value with specification below which contains a conditional logic would suit well for your case:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "*": {// values without ""(btw, name and age values are presumed to be non-null for every cases)
            "$": "[&3].&2"
          },
          "": "[&2].&1"// values with ""
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "age": "=toInteger"
      }
    }
  }
]

which converts all "" values to null for all attributes.

If conversion of only timestamp attributes, those have substring piece date , matters, then use the following spec

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": "[&1].&",
        "*date": {// to pick only timestamp attributes
          "*": {
            "$": "[&3].&2"
          },
          "": {
            "@": "[&3].&2"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "age": "=toInteger"
      }
    }
  }
]

the demo on the site http://jolt-demo.appspot.com/ is