如何将带有特殊字符（UTF-8）的HTML页面保存到txt文件

Question

I need to make a java code that save an html to a txt file.

The problem is that the special chars in UTF-8 are broken.

Words like "Hamamélis" are saved in this way "Hamam�lis".

the code that i writed is listed down there:

    URLConnection conn;
                    conn = site.openConnection();
                    conn.setReadTimeout(10000);
                    Charset charset = Charset.forName("UTF8");
                    BufferedReader in = new BufferedReader( new InputStreamReader(  conn.getInputStream(), "UTF-8"   )   );
                    buff = in.readLine();

And after:

out = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(Nome), "UTF-8"));
out.write(buff);
out.close();

Anyone can suggest me a solution?

Answer 1

One possible error is omitting the hyphen from "UTF-8" in the 4th line of your first piece of code. See the CharSet documentation.

Otherwise, code seems correct. But of course we cannot test it directly as we do not have your data.

For comparison, here is a little class I wrote. In a manner similar to your code, this class correctly writes your "Hamamélis" example's accented 'e' as the two octets expected in UTF-8 for a single (non-normalized) character: in hex 'C3' & 'A9'.

    import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStreamWriter;
import java.io.BufferedWriter;
import java.io.IOException;

public class ReaderWriter {
    public static void main(String[] args) {
        try {
            String content = "Hamamélis. Written: " + new java.util.Date();

            File file = new File("some_text.txt");

            // Create file if not already existent. 
            if (!file.exists()) {
                file.createNewFile();
            }

            FileOutputStream fileOutputStream = new FileOutputStream( file );
            OutputStreamWriter outputStreamWriter = new OutputStreamWriter( fileOutputStream, "UTF-8" );
            BufferedWriter bufferedWriter = new BufferedWriter( outputStreamWriter );
            bufferedWriter.write( content );
            bufferedWriter.close();

            System.out.println("ReaderWriter 'main' method is done. " + new java.util.Date() );

        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

As icktoofay commented, you should dig deeper to discover exactly what octets are involved. Use a hex editor like this " File Viewer " app I found today on the Mac App Store to see the exact octets in your saved file.

If the octets are C3 & A9, then the problem is simply that the text editor you used to look at the file as text used the wrong character encoding. For example, you can open that text file in a web browser, and use its menu commands to re-interpret the file as UTF-8.

If the octets are not C3 & A9, I would go further back to examine the input's octets.

If you do not understand that text files in computers actually contain numbers (not text in the human sense), then take a break from coding to read this entertaining article: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky