Parse CSV file containing a Unicode character using OpenCSV

Question

I'm trying to parse a .csv file with OpenCSV in NetBeans 6.0.1. My file contains some Unicode character. When I write it in output the character appears in other form, like (HJ1'-E/;). When when I open this file in Notepad, it looks ok.

The code that I used:

CSVReader reader=new CSVReader(new FileReader("d:\\a.csv"),',','\'',1);
    String[] line;
    while((line=reader.readNext())!=null){
        StringBuilder stb=new StringBuilder(400);
        for(int i=0;i<line.length;i++){
            stb.append(line[i]);
            stb.append(";");
        }
        System.out.println( stb);
    }

Answer 1

First you need to know what encoding your file is in, such as UTF-8 or UTF-16. What's generating this file to start with?

After that, it's relatively straightforward - you need to create a FileInputStream wrapped in an InputStreamReader instead of just a FileReader . ( FileReader always uses the default encoding for the system.) Specify the encoding to use when you create the InputStreamReader , and if you've picked the right one, everything should start working.

Note that you don't need to use OpenCSV to check this - you could just read the text of the file yourself and print it all out. I'm not sure I'd trust System.out to be able to handle non-ASCII characters though - you may want to find a different way of examining strings, such as printing out the individual values of characters as integers (preferably in hex) and then comparing them with the charts at unicode.org . On the other hand, you could try the right encoding and see what happens to start with...

EDIT: Okay, so if you're using UTF-8:

CSVReader reader=new CSVReader(
    new InputStreamReader(new FileInputStream("d:\\a.csv"), "UTF-8"), 
    ',', '\'', 1);
String[] line;
while ((line = reader.readNext()) != null) {
    StringBuilder stb = new StringBuilder(400);
    for (int i = 0; i < line.length; i++) {
         stb.append(line[i]);
         stb.append(";");
    }
    System.out.println(stb);
}

(I hope you have a try/finally block to close the file in your real code.)