用Jackson读取csv时的Java过滤器值

Question

I'm coming to you because I have a problem concerning the reading of a csv and the creation of a map with Jackson. In order to read the file and associate it to a map I use the following code:

private static <T, R extends IndexedData<T>> List<R> readFile(File csvFile, Class<R> cls) throws Exception {
    CsvMapper csvMapper = new CsvMapper();
    CsvSchema csvSchema = csvMapper.typedSchemaFor(cls).withHeader().withStrictHeaders(true);
    List list = csvMapper.readerFor(cls)
            .with(csvSchema.withColumnSeparator(';'))
            .with(CsvParser.Feature.SKIP_EMPTY_LINES)
            .readValues(csvFile)
            .readAll();
    return list;
}

however how to filter the data based on the value of an attribute?

For example, I have an ID in one column and I want to read and store the line in the map only if this value is equals to "038"

The code used reads the entire file and creates a HashMap with all the values, but I would like to add to the map only certain values ​​with defined criteria. Knowing that the files have a considerable volume, I can not read all the values ​​and filter the Map later.

Any help will be welcome :)

cordially

Answer 1

I am not sure it is possible to do it directly with the CsvMapper . An option would be to filter the lines before trying to deserialize them. Something like this:

I use an simple class and csv file here to give you an example

class MyClass {
    private String id;
    private String name;
    // Getters/Setters
}

and the following CSV file

id;name
1;pip
2;pap
3;pop
4;zip
5;zap
6;zop

You could do that: This example only deserialize the lines with a "o" in the name

public static void main(String[] args) throws Exception {
    readFile(new File("file.csv"));
}



private static List<MyClass> readFile(File csvFile) throws Exception {
    List<String> lines = filterFile(csvFile);

    CsvMapper csvMapper = new CsvMapper();
    CsvSchema csvSchema = csvMapper.typedSchemaFor(MyClass.class).withHeader().withStrictHeaders(true);

    List list = csvMapper.readerFor(MyClass.class)
            .with(csvSchema.withColumnSeparator(';'))
            .with(CsvParser.Feature.SKIP_EMPTY_LINES)
            .readValues(String.join("\n", lines))
            .readAll();

    return list;
}


// This method goes through the file and filter out all lines where the name does not contain an "o"
// It also keeps the header because it is needed for the CsvMapper
private static List<String> filterFile(File csvFile) {
    List<String> list = new ArrayList<>();
    boolean header = true;
    try (BufferedReader br = new BufferedReader(new FileReader(csvFile))) {
        String line;
        String[] tokens;
        while ((line = br.readLine()) != null) {
            tokens = line.split(";");
            if(header || tokens[1].contains("o")) {
                header = false;
                list.add(line);
            }
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
    return list;
}

Returns 2 MyClass objects with names "pop" and "zop"