how to skip blank lines while writing to csv file with Spring Batch FlatFileItemWriter

Question

I am trying to write the java data objects to.csv file using the Spring Batch FlatFifileItemWriter exactly as mentioned in the example at below location. http://www.mkyong.com/spring-batch/spring-batch-example-xml-file-to-csv-file/

The writer configuration is like below.

<bean id="cvsFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
    <!-- write to this csv file -->
    <property name="resource" value="file:cvs/report.csv" />
    <property name="shouldDeleteIfExists" value="true" />

    <property name="lineAggregator">
      <bean
        class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
        <property name="delimiter" value="," />
        <property name="fieldExtractor">
          <bean
            class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
            <property name="names" value="refId, name, age, csvDob, income" />
           </bean>
        </property>
       </bean>
    </property>
  </bean>

Since the FlatFileItemWriter has following write() method definition in the API, even though the LineAggregator returns empty String,it still writes to file.

StringBuilder lines = new StringBuilder();
        int lineCount = 0;
        for (T item : items) {
            lines.append(lineAggregator.aggregate(item) + lineSeparator);
            lineCount++;
        }
        try {
            state.write(lines.toString());
        }

is there any way to configure it to skip if the line is blank while writing to csv file in java using spring batch.

Answer 1

You'll want to add an ItemProcessor<YourObject> to your step. In it, you can add a simple YourObject process(YourObject item) method that checks if the object will result in a blank line. If not, return the object. If so, return null .

This will prevent the items you wish to discard from getting to the writer and will increase the FILTER_COUNT when you look at the BATCH_STEP_EXECUTION table for that Step.

More on filtering objects can be found in the Spring Batch docs.

Edit: The tutorial you referenced in your answer includes an example of a filtering processor ( FilterReportProcessor ) as well.

Answer 2

if memberclass is empty here's another way to writer using composite writer. Create another writer as below.

public class YourMemberFileItemWriter extends FlatFileItemWriter<YourMainClass>{
    @Override
    public String doWrite(List<? extends YourMainClass> items) {
        StringBuilder lines = new StringBuilder();
        for (YourMainClass item : items) {
            if ( YourMainClass.getYourMemberClass() != null) {
                lines.append(this.lineAggregator.aggregate(item)).append(this.lineSeparator);
            }
        }
        return lines.toString();
    }
}

Define your composite writer. CompositeItemWriter compositeItemWriter = new CompositeItemWriter<>(); compositeItemWriter.setDelegates(Arrays.asList( mainClassCsv(),memberClassCsv()); return compositeItemWriter;

Add member class methods as below to composite writer:

public FlatFileItemWriter memberClassCsv() {
    // TODO Auto-generated method stub
    YourMemberFileItemWriter fileWriter = new YourMemberFileItemWriter();
    fileWriter.setName("yourMemberClass");
    fileWriter.setResource(new FileSystemResource("memberfile.csv"));
    fileWriter.setLineAggregator(getMemberClassDelimitedLineAggregator());
    return fileWriter;
}

public DelimitedLineAggregator<YourMainClass> getMemberClassDelimitedLineAggregator() {
    BeanWrapperFieldExtractor<YourMainClass> beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<YourMainClass>();
    beanWrapperFieldExtractor.setNames(new String[] {
            "memberClassField.fieldname1"
            ,"memberClassField.fieldname2"
            });
    DelimitedLineAggregator<YourMainClass> delimitedLineAggregator = new DelimitedLineAggregator<YourMainClass>();
    delimitedLineAggregator.setDelimiter(",");
    delimitedLineAggregator.setFieldExtractor(beanWrapperFieldExtractor);
    return delimitedLineAggregator; 
}