将亚洲字符写入文件时出现 UnicodeEncodeError

Question

I've been running into this error rather consistently when trying to write Asian-characters to a file - I can print the characters just fine in IDLE (though not in the command line) but when I try to write them to a file, I get this error:

Traceback (most recent call last):
  File "C:\Users\Tai\Desktop\Development\playlistcreator\main.py", line 21, in <module>
    playlistcreator.addtoplaylist(list_of_paths,"C:\\Users\\Tai\\Desktop\\New Music\\testplaylist.m3u8")
  File "C:\Users\Tai\Desktop\Development\playlistcreator\playlistcreator.py", line 74, in addtoplaylist
    playlistFile.write(track + '\n')
  File "C:\Users\Tai\AppData\Local\Programs\Python\Python35-32\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 56-57: character maps to <undefined>

I think understand what the error is: basically it can't render Asian characters so it can't write them to a file - but if I can print them in IDLE, why wouldn't it be able to write them to a file? Is there any way to get around this?

(I'm not sure if the code is necessary to understand the error, but I'll post it anyway)

def addtoplaylist(paths, playlist_path):
    # Open file (or create if it does not exist)
    #change to given directory
    try:
        os.chdir(os.path.dirname(paths))
    except Exception as err:
        print(err)
    #open file - if it does not exist, create it
    with open(os.path.basename(playlist_path), 'a+') as playlistFile:
        for track in paths:
            playlistFile.write(track + '\n')
    playlistFile.close()

Answer 1

You need to specify the encoding for your file when opening it, for example:

with open(os.path.basename(playlist_path), 'a+', encoding='UTF-8') as playlistFile:

Otherwise Python defaults to using your console encoding (ANSI) for files, in this case CP 1252 which is the Windows name for extended Latin-1.

Answer 2

You need to use an encoding suitable for the characters.

playlistFile.write(track.encode(encoding=some_encoding) + '\n')

I guess some_encode = 'utf-8' will do but not really sure.

Maybe IDLE is using a different encoding by default and that's why it works there.