在CentOS 5.8中打开Python文件中的UnicodeEncodeError

Question

I have a problem with file open() in Python on 32 bit CentOS 5.8. When using characters outside of ascii in a filename I get the following error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 88: ordinal not in range(128)

The offending line is

file = open(full_path, 'w')

The code works fine on 64 bit Ubuntu 12.10 and 64 bit CentOS 6.3.

Some information from the server where the error happens:

$ python --version
Python 2.7.3
$ cat /etc/redhat-release 
CentOS release 5.8 (Final)
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

I have checked that on the Ubuntu machine the type of the "full_path" variable is 'unicode'. Part of the "full_path" variable comes from a configuration file in UTF-8 format. The rest from a web page in UTF-8 format.

I have not done any testing on the CentOS 5.8 machine yet as it is the production server I would rather not do random testing on it. Any hints to what can be the reason for this error is appreciated.

Update

If forgot to mention that the following works:

$ python
Python 2.7.3 (default, Jun 11 2012, 22:26:11) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> file = open(u'ø.txt', 'w')
>>>

Which in my head should be the same as the actual program, but apparently it isn't.

Update 2

Now it works for some reason. The program is a Turbogears web app and I just started and stopped the server daemon to see the result of some debug printing and the error no longer happens.

I am still at loss for why this happens, but at least the problem is temporarily resolved.

Answer 1

I'm not sure if this will work as a solution, but i had a error like that. I solved it, but i have some side effects now (look at my profile). Try:

import sys
reload(sys)
sys.setdefaultencoding('utf-8')