Unicode working on Windows but not Red Hat Linux : Java

Question

In Java, I am generating file having Unicode characters.

When I run my program in Windows (Jboss) and open the file (CSV). It finely displays Unicode characters (Norwegin and Icelandic) in excel.

But when I deploy the same in server inside Red Hat Linux (in Jboss same version), run the program, generate file and download and when I see that in excel then it is distorting all Unicode characters.

Could you please suggest any local Linux setting due to which Unicode is distorting? or where change is required?

 FileWriter writer = new FileWriter(fileName);
 writer.append(new String(data.toString().getBytes("UTF-8"),"UTF-8"));
 writer.flush();
 writer.close();

 //data is StringBuilder type

I have also tried ISO8859_1

Update 1

I have checked System Encoding: using System.getProperty("file.encoding") and found that

Windows is Cp1252 and Linux is UTF-8

Update 2

When I print in Linux using :

log.info(new String(data.toString().getBytes("UTF-8"), "UTF-8"));

it is showing all output perfectly fine but when I put it in FileWriter with extension filename.csv, it is not displaying correctly.

Answer 1

It looks like you are translating from bytes

data

to String

data.toString()

to Bytes

data.toString().getBytes("UTF-8")

to String

new String(data.toString().getBytes("UTF-8"),"UTF-8"))

to bytes

writer.append(new String(data.toString().getBytes("UTF-8"),"UTF-8"));

Try just a single conversion from the input encoding to a String and then write out the String. So the data.toString() needs to know what encoding it is reading. Does data support conversion from different code pages?

writer.append(data.toString(codepage));