Unable to Parse Final Column from CSV File using Apache Commons CSV

Question

I have a CSV file with 19 columns in it (the last column is empty). I'm parsing the file via Apache Commons CSV. When I execute:

System.out.println("csvRecord.toString() = " + csvRecord.toString());

I get the following output:

csvRecord.toString() = CSVRecord [comment=null, mapping={Amount=6, ID=2, UID=4, Address=5, Carrier=17, CPL=14, F=12, GC=13, PIT=9, PP=11, PCIT=10, PT=1, SPI=3, SPI1=8, ST=7, Status=16, SOI=0, TN=18, TP=15}, recordNumber=6, values=[FB, S13 Wood Carving, B07, AG, bis, FB, 44.8, PE, AG XXXXXXX47, UNKNOWN, 39.95, 1.6, 4.1, , -2.2068006148899633, -0.46425251802390777, COMPLETED, US]]

As you can see, the last column TN=18 (which is actually column 19 when counting from one) is included in output of toString() method in the mapping section (but not in the values section).

However, when I convert csvRecord to map via the following code:

Map<String, String> csvRecordMap = csvRecord.toMap();

The last column is no longer present. And when I execute:

if (csvRecordMap.containsKey("TN")) 

It returns false.

Based on my testing I've noticed that the toMap() method will not include an empty column if there are no non empty columns after it (even if the column is actually present in CSV file). If a column is empty but there are non empty columns after it, then it will be included in return value of toMap() method.

How can I force parser to return all columns -- either via toMap() method or by somehow extracting it from csvRecord directly as it is obviously present as shown in toString() method?

Thanks!

Answer 1

Loop through and add the missing names:

Map<String, String> map = csvRecord.toMap();
for (String name : csvParser.getHeaderNames())
    map.putIfAbsent(name, "");

Demo

CSVFormat csvFormat = CSVFormat.DEFAULT.withHeader();
try (CSVParser csvParser = csvFormat.parse(Files.newBufferedReader(Paths.get("test.txt")))) {
    System.out.println(csvParser.getHeaderNames());
    for (CSVRecord csvRecord : csvParser) {
        System.out.println(csvRecord);
        System.out.println("  toMap(): " + csvRecord.toMap());

        Map<String, String> map = csvRecord.toMap();
        for (String name : csvParser.getHeaderNames())
            map.putIfAbsent(name, "");
        System.out.println("  fixed  : " + map);
    }
}

test.txt

A,B,C,D
1,2,3,4
1,2,3
1,2
1
1,
1,,
1,,,
,,,4,,,

Output (with commons-csv-1.7.jar )

[A, B, C, D]
CSVRecord [comment='null', recordNumber=1, values=[1, 2, 3, 4]]
  toMap(): {A=1, B=2, C=3, D=4}
  fixed  : {A=1, B=2, C=3, D=4}
CSVRecord [comment='null', recordNumber=2, values=[1, 2, 3]]
  toMap(): {A=1, B=2, C=3}
  fixed  : {A=1, B=2, C=3, D=}
CSVRecord [comment='null', recordNumber=3, values=[1, 2]]
  toMap(): {A=1, B=2}
  fixed  : {A=1, B=2, C=, D=}
CSVRecord [comment='null', recordNumber=4, values=[1]]
  toMap(): {A=1}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=5, values=[1, ]]
  toMap(): {A=1, B=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=6, values=[1, , ]]
  toMap(): {A=1, B=, C=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=7, values=[1, , , ]]
  toMap(): {A=1, B=, C=, D=}
  fixed  : {A=1, B=, C=, D=}
CSVRecord [comment='null', recordNumber=8, values=[, , , 4, , , ]]
  toMap(): {A=, B=, C=, D=4}
  fixed  : {A=, B=, C=, D=4}