如何在Mule Dataweave中读取带有空行和多余标题的CSV文件

Question

I have CSV having blank line and extra herder below that like this:

**blank line**
Available_Date_Feed
productID,availableDate
148305801,2015-08-07T00:00:00.000+0000
160611862,2015-07-29T00:00:00.000+0000
160611715,2015-07-29T00:00:00.000+0000
160342798,2015-07-29T00:00:00.000+0000

I want to read values of productID & availableDate. If we do regular transformation with dataweave it will return null values

This is the code I have wrote in dataweave:

%dw 1.0
%input in0 application/csv headers=true
%output application/java
---
payload  map  {
    productID:$.productID,
    availableDate:$.availableDate
}

Returning payload as :

[{productID=null, availableDate=null}, {productID=null, availableDate=null}, {productID=null, availableDate=null}, {productID=null, availableDate=null}]

Any suggestions here? Can we use Groovy/MEL/regex expression for this? How to use Rows to ignore in Dataweave?

Can we Skip first 2 lines using groovy/regex?

I am facing performance issue with the below groovy. Mule taking too much time to convert even 1 MB file. Is there any another solution for this?

Answer 1

I have used Groovy to skip the first 2 lines. This script fetch the comma separated values directly

 csvContent = message.payload
def filteredContent = new StringBuffer()
regexPattern = /(\S*),(\S*)/
finder = csvContent =~ regexPattern

(0..<finder.count).each {
    //println "Iteration: ${it+1}:"
    filteredContent.append(finder[it][0])
    filteredContent.append('\n')

}

return filteredContent.toString()

use object to string before groovy

Answer 2

I know it's late. If still waiting for answer, try this. It's done in dataweave only.

%dw 1.0
%output application/csv header=false
---
payload[3..-1] map {
    productId:$[0],
    date: $[1]
}

Hope it helps.

Answer 3

This link might helpful

https://developer.mulesoft.com/docs/dataweave

and read about below code snippet in above doc to skip the null values

%output application/xml skipNullOn="everywhere"