如何正确地解码和重新编码来自latin-1的文件名以打开它

Question

I am trying to open a local file in selenium using

driver.get( ('file://' + file ))

where file is the file name.

It seems the file name has latin-1 characters in it:

..\\\\PRODUCT NAME – Something Something.html

when I use file.decode('latin-1') , I get:

..\\\\PRODUCT NAME \\x96 Something Something.html

If I simply use driver.get( ('file://' + file )) , I get:

UnicodeDecodeError: 'utf8' codec can't decode byte 0x96 in position 81: invalid start byte

If I use driver.get( ('file://' + file.decode('latin_1') )) , I get an error stating that the file is not found:

...fileNotFound&u=file%3A///C%3A/PRODUCT%20NAME%20%C2%96%20Something%20Something.html.

I'm not 100% sure what encoding it's expecting, but I've tried re-encoding the file name as unicode and utf-8 with no luck (same error - says file isn't found).

Any idea how I could solve this problem? Renaming the file itself won't be an option unfortunately. I want to properly decode it, then re-encode it (the encoding sandwich others have recommended).

Answer 1

Figured it out. The problem is that the html document has charset='utf-8' , and it isn't strictly true. The title contains ' – ', which is cp1252 encoded.

I've solved it like this by adding an exception and character correcting function:

def selenium_extractor(file):

    def charset_correct(filename):
        try:
            return ''.join([char.decode('cp1252') for char in filename])
        except:
            # EXTEND WITH FURTHER DECODINGS
            traceback.print_exc()
            sys.exit(1)


    driver = webdriver.Firefox()

    try:
        driver.get( 'file://' + file)
        driver.quit()

    except UnicodeError:
        driver.get( 'file://' + charset_correct(file))
        driver.quit()