Django: Non-ASCII character

Question

My Django View/Template is not able to handle special characters. The simple view below fails because of the ñ. I get below error:

Non-ASCII character '\\xf1' in file"

def test(request):
    return HttpResponse('español')

Is there some general setting that I need to set? It would be weird if I had to handle all strings separately: non-American letters are pretty common!

EDIT This is in response to the comments below. It still fails :(

I added the coding comment to my view and the meta info to my html, as suggested by Gabi.

Now my example above doesn't give an error, but the ñ is displayed incorrectly.

I tried return render_to_response('tube/mysite.html', {"s": 'español'}) . No error, but it doesn't dislay (it does if s = hello). The other information on the html page displays fine.

I tried hardcoding 'español' into my HTML and that fails:

UnicodeDecodeError 'utf8' codec can't decode byte 0xf.

I tried with the u in front of the string:

SyntaxError (unicode error) 'utf8' codec can't decode byte 0xf1

Does this help at all??

Answer 1

Do you have this at the beginning of your script:

# -*- coding: utf-8 -*-

...?

See this: http://www.python.org/dev/peps/pep-0263/

EDIT : For the second problem, it's about the html encoding. Put this in the head of your html page (you should send the request as an html page, otherwise I don't think you will be able to output that character correctly):

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Answer 2

Insert at the top of views.py

# -*- coding: utf-8 -*-

And add "u" before your string

my_str = u"plus de détails"

Solved!

Answer 3

You need the coding comment Gabi mentioned and also use the unicode "u" sign before your string :

return HttpResponse(u'español')

The best page I found on the web explaining all the ASCII/Unicode mess is this one : http://www.stereoplex.com/blog/python-unicode-and-unicodedecodeerror

Enjoy!

Answer 4

在settings.py文件中将DEFAULT_CHARSET设置为'utf-8' 。

Answer 5

ref from: https://docs.djangoproject.com/en/1.8/ref/unicode/

"If your code only uses ASCII data, it's safe to use your normal strings, passing them around at will, because ASCII is a subset of UTF-8.

Don't be fooled into thinking that if your DEFAULT_CHARSET setting is set to something other than 'utf-8' you can use that other encoding in your bytestrings! DEFAULT_CHARSET only applies to the strings generated as the result of template rendering (and email). Django will always assume UTF-8 encoding for internal bytestrings. The reason for this is that the DEFAULT_CHARSET setting is not actually under your control (if you are the application developer). It's under the control of the person installing and using your application – and if that person chooses a different setting, your code must still continue to work. Ergo, it cannot rely on that setting.

In most cases when Django is dealing with strings, it will convert them to Unicode strings before doing anything else. So, as a general rule, if you pass in a bytestring, be prepared to receive a Unicode string back in the result."

Answer 6

The thing about encoding is that apart from declaring to use UTF-8 (via <meta> and the project's settings.py file) you should of course respect your declaration: make sure your files are saved using UTF-8 encoding.

The reason is simple: you tell the interpreter to do IO using a specific charset. When you didn't save your files with that charset, the interpreter will get lost.

Some IDEs and editors will use Latin1 (ISO-8859-1) by default, which explains why Ryan his answer could work. Although it's not a valid solution to the original question being asked, but a quick fix.

Answer 7

I was struggling with the same issue as @dkgirl, yet despite making all of the changes suggested here I still could not get constant strings that I'd defined in settings.py that contain ñ to show up in pages rendered from my templates.

Instead I replaced every instance of "utf-8" in my python code from the above solutions to " ISO-8859-1 " (Latin-1). It works fine now.

Odd since everything seems to indicate that ñ is supported by utf-8 (and in fact I'm still using utf-8 in my templates). Perhaps this is an issue only on older Django versions? I'm running 1.2 beta 1.

Any other ideas what may have caused the problem? Here's my old traceback:
Traceback (most recent call last):
File "manage.py", line 4, in
import settings # Assumed to be in the same directory.
File "C:\\dev\\xxxxx\\settings.py", line 53
('es', ugettext(u'Espa±ol') ),
SyntaxError: (unicode error) 'utf8' codec can't decode byte 0xf1 in position 0: unexpected end of data