简体   繁体   中英

Shift JOLT transformation - facing problem with below transformation

I'm trying to convert below input json to flatten necessary column names and its values while retaining all metadata.

Below is the input json that I've for my CDC use-case.

{
  "type": "update",
  "timestamp": 1558346256000,
  "binlog_filename": "mysql-bin-changelog.000889",
  "binlog_position": 635,
  "database": "books",
  "table_name": "publishers",
  "table_id": 111,
  "columns": [
    {
      "id": 1,
      "name": "id",
      "column_type": 4,
      "last_value": 2,
      "value": 2
    },
    {
      "id": 2,
      "name": "name",
      "column_type": 12,
      "last_value": "Suresh",
      "value": "Suresh123"
    },
    {
      "id": 3,
      "name": "email",
      "column_type": 12,
      "last_value": "Suresh@yahoo.com",
      "value": "Suresh@yahoo.com"
    }
  ]
}

Below is the expected output json

[
  {
    "type": "update",
    "timestamp": 1558346256000,
    "binlog_filename": "mysql-bin-changelog.000889",
    "binlog_position": 635,
    "database": "books",
    "table_name": "publishers",
    "table_id": 111,
    "columns": {
      "id": "2",
      "name": "Suresh123",
      "email": "Suresh@yahoo.com"
    }
  }
]

I tried the below spec from which I'm able to retrieve columns object but not the rest of the metadata.

[
  {
    "operation": "shift",
    "spec": {
      "columns": {
        "*": {
          "@(value)": "[#1].@(1,name)"
        }
      }
    }
  }
]

Any leads would be very much appreciated.

I got the JOLT spec for above transformation. I'm posting it here incase if anyone stumbles upon the something like this.

[
  {
    "operation": "shift",
    "spec": {
      "columns": {
        "*": {
          "@(value)": "columns.@(1,name)"
        }
      },
      "*": "&"
    }
  }
]

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM