Java application reads UTF-8 encoded text files but after ant build the characters are not as expected

Question

I have a Java project that reads UTF-8 encoded .txt files in order to construct strings that are displayed. Running in Eclipse the files are read and displayed as expected but after my ant build the characters are not coming out as they should.

Here is my build.xml file

<?xml version="1.0"?>
<project name="Rutherford" default="jar">

    <property name="libsSrc" value="libs"/>
    <property name="build" value="build"/>
    <property name="classes" value="build/classes"/>
    <property name="jar" value="build/jar"/>
    <property name="libs" value="build/libs"/>

    <path id="classpath">
            <fileset dir="${libsSrc}" includes="*.jar"/>
    </path>

    <pathconvert property="mf.classpath" pathsep=" ">
            <path refid="classpath"/>
            <mapper>
                    <chainedmapper>
                            <flattenmapper/>
                            <globmapper from="*.jar" to="lib/*.jar"/>
                    </chainedmapper>
            </mapper>
    </pathconvert>

    <target name="clean" description="remove intermediate files">
        <delete dir="build"/>
    </target>

    <target name="compile" description="compile the Java source code to class files">
        <mkdir dir="${classes}"/>
        <javac srcdir="." destdir="${classes}" classpathref="classpath">
            <compilerarg line="-encoding utf-8"/>
        </javac>
    </target>

    <target name="jar" depends="compile" description="create a Jar file for the application">
        <mkdir dir="${jar}"/>
        <jar destfile="${jar}/App.jar">
            <zipgroupfileset dir="${libsSrc}" includes="*.jar"/>
            <fileset dir="${classes}" includes="**/*.class"/>
            <manifest>
                <attribute name="Main-Class" value="nat.rutherford.DesktopStarter"/>
                <attribute name="Class-Path" value="${mf.classpath}"/>
            </manifest>
        </jar>
    </target>

</project>

I tried to compile with UTF-8 character encoding using

<compilerarg line="-encoding utf-8"/>

but obviously this is not enough, what do I need to modify to get this to work?

Thanks

Edit 1

I read the .txt files in with.

public String fileToString(String file) {
    InputStream in = new BufferedInputStream(Gdx.files.internal(file).read());
    return new Scanner(in).useDelimiter("\\A").next();
}

which returns a string to a String variable, then I simply take this string and pass it to an object

It works ok when compiling and running within eclipse.

Edit 2

This is the fix

public String fileToString(String file) {
    InputStream in = new BufferedInputStream(Gdx.files.internal(file).read());
    return new Scanner(in, "UTF-8").useDelimiter("\\A").next();
}

Simple when you know where to look! lol Thanks!

Answer 1

As per The documentation , use encoding='utf-8' instead of compiler-arg .

However ... I suspect that your problem is runtime, not compile time. Do you create OutputStreamWriter objects without passing an encoding? Or pass interesting chars to System.out.println ? Or correspondingly InputStreamReader or Scanner ? The fix in this case would be to add -Dfile.encoding=utf-8 to the command line.