非ASCII字符'\\ xc5'破坏python脚本

Question

My node.js server outputs the following array:

["Daniel Guillen","Sarah Tremaine Milam","Karen Ann"...."Chris Doyle","Katie Gould"]

I'm trying to convert it to a csv file using a python script but when I declare the array in my .py file I get the following error:

SyntaxError: Non-ASCII character '\xc5' in file toCSV.py

The array consists of people's names from Facebook so someone somewhere has a weird character in their name. It is too long to manually search.

How do you recommend working around this problem or finding the offending character? Iterating through the array in javascript before handing to python is also an option.

Answer 1

If the array is embedded in the python script then you may need to set the encoding at the start of the python file. To do this add the following line before any code:

# -*- coding: utf-8 -*-

This sets the source code encoding to UTF-8.

Answer 2

You can use codecs (you'll have to import it) to read utf8 (non-ascii) characters. Like so:

csvfile = codecs.open(outputFileName+".csv", 'w+', 'utf8')

That would be for writing to an output file. If you want to read in a file, just change the 'w+' to 'r'. For individual strings, you can also use the method "string".encode('utf8') This fixes the issue if it's being read from another file. If you have it in the file you're running, you need to set the encoding by putting

# -*- coding: utf-8 -*- 

at top of file