Dataweave 2.0 array filtering based on another arrays values

Question

I'm attempting to try and match/filter the following lineId values in the unmatchedIds array to then filter the result set of exampleFile, by processorTransactionId. The result would be the removal of the last financialTransactionEntity, with the processorTransactionId = "000000062121029333".

In theory the sizes of both the unmatchedIds array and exampleFile array could be unbounded.

Any guidance/advice/examples would be much appreciated. I'm having difficult using the dataweave filter to achieve this at the moment.

{
  "unmatchedIds": [
    {
      "lineId": "000000062121029111"
    },
    {
      "lineId": "000000062121029222"
    }
  ]
}

exampleFile

[{
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "Burger Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000001000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029111"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029222"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029333"
        }
    }

]

Answer 1

You can make use of data selector with filter. See below

%dw 2.0
output application/java

import * from dw::core::Arrays

var unMatchedIds = {
  "unmatchedIds": [
    {
      "lineId": "000000062121029111"
    },
    {
      "lineId": "000000062121029222"
    }
  ]
}

var payload = [{
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "Burger Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000001000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029111"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029222"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029333"
        }
    }

]

---

using (unMatchedArray = unMatchedIds.unmatchedIds.*lineId)
payload[?(unMatchedArray contains $.financialTransactionEntity.processorTransactionId)]

unMatchedArray here is just a list of unmatched ids coming from your original object (unMatchedIds). Data selector will just include entry if financialTransactionEntity.processorTransactionId is in the unMatchedArray.

Answer 2

You can use groupBy and filter functions to achieve the result. First, you can group unMatchIds by lineId and then you can apply a filter on your actual payload to eliminate all unmatched items.

output application/json
var unMatchIdsGrouped = var.unMatchIds.unmatchedIds groupBy $.lineId
---
payload filter ((item) -> unMatchIdsGrouped[ item.financialTransactionEntity.processorTransactionId] !=null)

Answer 3

I would suggest transforming the unmatchedIds object to an array of id 's, then filtering exampleFile objects based on whether or not the processorTransactionId is found in the compared array using contains full script:

%dw 2.0
output application/java


var unMatchedIds = {
  "unmatchedIds": [
    {
      "lineId": "000000062121029111"
    },
    {
      "lineId": "000000062121029222"
    }
  ]
}

var exampleFile = [{
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "Burger Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000001000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029111"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029222"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029333"
        }
    }

]

---
exampleFile filter (unMatchedIds.unmatchedIds.lineId contains $.financialTransactionEntity.processorTransactionId)

Answer 4

Here's another answer ....Hope it helps

    %dw 2.2
output application/json
var inpUnmatchedIds = {
  "unmatchedIds": [
    {
      "lineId": "000000062121029111"
    },
    {
      "lineId": "000000062121029222"
    }
  ]
}
var exampleIds = [{
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "Burger Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000001000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029111"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029222"
        }
    },
    {
        "financialTransactionEntity": {
            "cardAcceptor": {
                "name": "McDonalds Inc.",
                "countryCode": "GBP"
            },
            "financialTransaction": {
                "debitOrCredit": "C",
                "amountInOriginalCurrency": {
                    "amount": "0000002000",
                    "exponent": "2"
                },
                "originalCurrencyCode": "826",
                "transactionDate": "2020-02-18"
            },
            "processorTransactionId": "000000062121029333"
        }
    }
]
---
exampleIds filter (value) -> ((((inpUnmatchedIds pluck $.lineId))[0]) contains (value.financialTransactionEntity.processorTransactionId))