which data structure should I use to fetch and keep data from a CSV file or a tab seperated value file in java

Question

I have a CSV/tab seperated file with basically have data of a table, I need to fetch it into java and later use it for comparison with a table in database. I have actually done this before using a 2d array but I think that is not very efficient when the file size increases. And also any operations in a 2d array requires a lot looping and conditions. So which data structure should I prefer in java or 2d array is just fine ?

Answer 1

I think that if you need to work on columns, a List of Maps could do the job:

List<Map<String, String>> table = new ArrayList<Map<String, String>>();

Then, every row can be a Map<String, String> , with the key being the name of the column and the value being the actual value.

for (int i = 0; i < TABLE_ROWS; i++)
   Map<String, String> row = new HashMap<String, String>();
   row.put("column1", "value 1");
   row.put("column2", "value 2");
   ...
   table.add(row);
}

So, when you have to work with a specific value you can do like this:

table.get(ROW_NUMBER).get(COLUMN_NAME);

As an alternative, you can write a POJO that would be used as a model for the rows, as suggested by YoungHobbit, and use that POJO instead of the Map.

Answer 2

You can define an POJO class for the attributes in the CSV file. Use jackson-csv library for reading the data from csv file and creating object of your custom type.

If the attributes of the database also same as the csv file then create the object out of it. In that case you can compare these two objects using equals method, which you need to override for your needs.

Otherwise you can write different implementation for comparing them.

Answer 3

I believe you can either use a POJO as suggested by @YoungHobbit, but in that case you will have to be sure that there will be certain fields in each row. There can be less but not new columns.

OR

You can use a HashMap<String, List<String>>. That way you can use a unique field as Key and rest of the fields as value in a list.

Answer 4

The right answer is : it depends on what you want to do with it.

The generic answer would be

List<List<String>> csv = new ArrayList<>();

And create a new ArrayList for each line.

String cell = csv.get(row).get(column);

If you have a lot of holes in your CVS, you could use a comparable object as coordinate.

public class Coordinate implements Comparable<Coordinate> {
    public int row;
    public int column;
    public Coordinate(int r, int c) {
        row = r;
        column = c;
    }

    @Override
    public int compareTo(Coordinate o) {
        int r = Integer.compare(row, o.row);
        if(r == 0) {
            r = Integer.compare(column, o.column);
        }
        return r;
    }

    public boolean equals(Object o) {
        if(o instanceof Coordinate) {
            Coordinate c = (Coordinate)o;
            return row == c.row && column == c.column;
        }
        return false;
    }
}

Then use a TreeMap :

    TreeMap<Coordinate, String> csv = new TreeMap<>();

    csv.put(new Coordinate(1,2), "Hello");

    String cell = csv.get(new Coordinate(3,4));

    for(Map.Entry<Coordinate, String> e : csv.entrySet()) {
        Coordinate c = e.getKey();
        String cell = e.getValue();
    }

    for(String cel : csv.values()){
        //...
    }