没有http-equiv元标记且字符集的HTML文件可能不是UTF-8

Question

we are using jsoup - excellent thanks.

We may get HTML files with no http-equiv meta tag and the charset may be other than UTF-8. How is it best to handle this please. We can have a list of encodings and try them but I am not sure how to tell programatically if something is wrong. Would jsoup throw an IOException?

Answer 1

Jsoup will try to determine the encoding by the content type header or http equiv tag , if you have none of them it will use utf8 . Not sure if jsoup can do more for you here.

But you can try another approach:

Implement a class that reads the files for you. There you can take care of all encoding issues. As a result such a class should give you proper encoded string or at least the encoding that's used for your input.

(html input) --> [encoding class] --normalized encoding--> [jsoup] --> (whatever)   

Jsoup can now parse that input with a known encoding.

I guess changes on the html-creation thing is not possible, isn't it?

Some further readings:

• http://illegalargumentexception.blogspot.co.uk/2009/05/java-rough-guide-to-character-encoding.html#javaencoding_autodetect
• Character Encoding Detection Algorithm
• What is the most accurate encoding detector? (includes a list of implementation)
• Java Text File Encoding
• Detect (or best guess of) incoming string encoding in Java