How to fix ApplicationResources_fr.properties getting corrupted

Question

I have a problem in a class I wrote. The purpose of the class is to add/remove/update for the applicationResource.properties files which the <spring:message code="key" /> uses to provide bilingual support to the website. Manually interacting with the properties files works fine, but I had a greater need and so I built in a way to allow changes to be done from the database. This has given me a very dynamic and flexible system that I can work from.

However, there is a problem. At some point after even a single change using this, the French characters end up getting changed. Such as Déconnexion becoming Déconnexion . When looked at in notepad++ its first Déconnexion and then corrupted to D\Ã\©connexion . This example was part of the original properties file.

The original (not temp) properties files have the text file encoding set to other: UTF-8 . The Project properties text file encoding is set to inherited from container (Cp1252) . I tried changing to Other: UTF-8 with no change.

So my question(s) is, what is causing the corruption to my French characters and how can I fix it? I have provided the complete class below.

Update: After the assistance from StephaneM in her answer I was able to track down exactly what is causing the corruption, but have not fixed it yet. The loadProperties() function in the AR Class. As soon as the temp AP files are loaded the French characters are corrupted. This makes me suspect the original process which creates the temp AP files is using a different encoding. So I will have to track it down.

package pojo;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Objects;
import java.util.Properties;
import java.util.Set;

import org.springframework.beans.factory.annotation.Autowired;

/*
 * Purpose of this class is to handle all the     ApplicationResource(_fr).properties interactions
 * so that there is one unified location handling this, instead of code duplication.
 */

public class AR{
public final String en_path = "/ApplicationResources.properties";
public final String fr_path = "/ApplicationResources_fr.properties";

private Properties en_prop = null;
private Properties fr_prop = null;

public AR()
{
    loadProperties();
}
private void loadProperties()
{
    InputStream en_is = null;
    InputStream fr_is = null;
    try {
        this.en_prop = new Properties();
        this.fr_prop = new Properties();
        en_is = this.getClass().getResourceAsStream(en_path);
        fr_is = this.getClass().getResourceAsStream(fr_path);
        en_prop.load(en_is);
        fr_prop.load(fr_is);
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
}
private boolean keyExist(String mykey, String mypath)   //deprecated due to better code/method
{
    Properties test_prop = null;
        InputStream is = null;
        try {
            test_prop = new Properties();

            is = this.getClass().getResourceAsStream(mypath);
            test_prop.load(is);
            Set<Object> keys = test_prop.keySet();
            for(Object k:keys) {
                String key = (String)k;
                //System.out.print(key + " ");
                if(key.equals(mykey))   
                {
                    return true;
                }
            }
            //System.out.println(" ");
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (NullPointerException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    return false;
}
public boolean en_keyExist(String mykey)
{
    //searches english file
    loadProperties();
    return en_prop.containsKey(mykey);
    //return keyExist(mykey, en_path);  //original method
}
public boolean fr_keyExist(String mykey)
{
    //searches french file
    loadProperties();
    return fr_prop.containsKey(mykey);
    //return keyExist(mykey, fr_path);  //original method
}
public boolean en_fr_keyExist(String mykey)
{
    //searches both english and french files
    loadProperties();
    return (en_prop.containsKey(mykey) && fr_prop.containsKey(mykey));
    //return (keyExist(mykey, en_path) && keyExist(mykey, fr_path));    //original method
}
public String en_returnProperty(String mykey)
{
    //returns null if key does not exist
    loadProperties();
    return this.en_prop.getProperty(mykey);
}
public String fr_returnProperty(String mykey)
{
    //returns null if key does not exist
    loadProperties();
    return this.fr_prop.getProperty(mykey);
}
public void appendProperty(Properties new_en_prop,Properties new_fr_prop)
{
    //note: during a test, setProperty (used in populating the properties) does not allow duplicates, it overwrites.
    //So, load the existing properties, and for each new property add it

    loadProperties();
    for(Object key : new_en_prop.keySet())
    {
        en_prop.setProperty((String)key, new_en_prop.getProperty((String)key));
    }
    try (OutputStream en_os = new FileOutputStream(getClass().getResource(en_path).getFile(),false);)
    {
        en_prop.store(en_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }
    for(Object key : new_fr_prop.keySet())
    {
        fr_prop.setProperty((String)key, new_fr_prop.getProperty((String)key));
    }
    try (OutputStream fr_os = new FileOutputStream(getClass().getResource(fr_path).getFile(),false);)
    {
        fr_prop.store(fr_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }

}
public boolean appendProperty(String mykey, String en_val, String fr_val)   //appears to have timing error due to only saving last value
//due to timing error this function is only suitable for single additions
//due to the timing error, tried returning boolean to have it finished but was not successful
//setting the class variables to static did not solve the timing issue
{
    loadProperties();
    en_prop.setProperty(mykey, en_val);
    try (OutputStream en_os = new FileOutputStream(getClass().getResource(en_path).getFile(),false);)
    {
        en_prop.store(en_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }
    fr_prop.setProperty(mykey, fr_val);
    try (OutputStream fr_os = new FileOutputStream(getClass().getResource(fr_path).getFile(),false);)
    {
        fr_prop.store(fr_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return true;

}
public void en_setProperty(String mykey, String en_val)
//suspected timing issue, use only for singular changes
{
    loadProperties();
    en_prop.setProperty(mykey, en_val);
    try (OutputStream en_os = new FileOutputStream(getClass().getResource(en_path).getFile(),false);)
    {
        en_prop.store(en_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }
}
public void fr_setProperty(String mykey, String fr_val)
//suspected timing issue, use only for singular changes
{
    loadProperties();
    fr_prop.setProperty(mykey, fr_val);
    try (OutputStream fr_os = new FileOutputStream(getClass().getResource(fr_path).getFile(),false);)
    {
        fr_prop.store(fr_os, null);
    } catch (IOException e) {
        e.printStackTrace();
    }
}
public void compareResources()
{
    Properties new_en = new Properties();
    Properties new_fr = new Properties();

    for(Object key : en_prop.keySet())
    {
        new_en.setProperty((String)key, en_prop.getProperty((String)key));
    }
    for(Object key : fr_prop.keySet())
    {
        new_fr.setProperty((String)key, fr_prop.getProperty((String)key));
    }

    Properties temp = (Properties) new_en.clone();

    for(Object key : temp.keySet())
    {
        if(new_fr.containsKey((String) key))
        {
            new_fr.remove(key);
            new_en.remove(key);
        }
    }

    for(Object key : new_en.keySet())
    {
        System.out.println("English only key: " + ((String)key));
    }
    for(Object key : new_fr.keySet())
    {
        System.out.println("French only key: " + ((String)key));
    }
}   

}

Sample use case for the class, taken directly from application, but with some editing so only the relevant parts are here

AR testing = new AR();
Properties en_prop = new Properties();
Properties fr_prop = new Properties();

final String test_prod_cur = "{call BILINGUAL_VALUES(?)}";
ResultSet rs = null;
try     ( 
  Connection connection = jdbcTemplate.getDataSource().getConnection();
  CallableStatement callableStatement = connection.prepareCall(test_prod_cur);
)
{
    callableStatement.registerOutParameter(1, OracleTypes.CURSOR);
    callableStatement.executeUpdate();
    rs = (ResultSet) callableStatement.getObject(1);
    while (rs.next())
    {
        String thead = rs.getString(1);
        en_prop.setProperty(keyheader+thead, rs.getString(2));
        fr_prop.setProperty(keyheader+thead, rs.getString(3));
        //testing.appendProperty(keyheader+thead, rs.getString(2), rs.getString(3));    //has a timing issue, ends up only appending final value
   }
}
catch (SQLException e)
{
System.out.println("SQLException - bilingual values");
System.out.println(e.getMessage());
}       
testing.appendProperty(en_prop, fr_prop);

Answer 1

Regarding this question: "what is causing the corruption to my French characters and how can I fix it?", the answer is in the documentation ( Properties.store() ):

public void store(OutputStream out, String comments) throws IOException

Writes this property list (key and element pairs) in this Properties table to the output stream in a format suitable for loading into a Properties table using the load(InputStream) method.

Properties from the defaults table of this Properties table (if any) are not written out by this method.

This method outputs the comments, properties keys and values in the same format as specified in store(Writer), with the following differences:

• The stream is written using the ISO 8859-1 character encoding.
• Characters not in Latin-1 in the comments are written as \\uxxxx for their appropriate unicode hexadecimal value xxxx.
• Characters less than \ and characters greater than \~ in property keys or values are written as \\uxxxx for the appropriate hexadecimal value xxxx.

Answer 2

I am not the only person who has faced this issue, I managed to find another question and it was one of the answers that lead me to my solution. I have to thank another site for letting me know what to include.

There are only four added or changed lines, I will list them, and then give a complete function.

import java.io.Reader;
Reader reader = new InputStreamReader(fr_is, "UTF-8");
fr_prop.load(reader); //instead of fr_prop.load(fr_is);
reader.close();

The complete function

import java.io.Reader;

private void loadProperties()
{
    InputStream en_is = null;
    InputStream fr_is = null;
    try {
        this.en_prop = new Properties();
        this.fr_prop = new Properties();
        en_is = this.getClass().getResourceAsStream(en_path);
        fr_is = this.getClass().getResourceAsStream(fr_path);

        Reader reader = new InputStreamReader(fr_is, "UTF-8");

        en_prop.load(en_is);
        fr_prop.load(reader);
        reader.close();

    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

By introducing the reader and using that, it has cleared up French character corruption.

I should mention that I changed every file property I could find over to UTF-8 before I did the above changes and got it working. This site gives you the changes I made. This was a nice page that helped me confirm the encodings.