UnicodeDecodeError in django top-level template code

Question

I have code in my views that returns information to be displayed in a textbox. My name has fadas (Irish accent) over the letters which is causing UnicodeDecodeErrors. The line in my logic is as follows:

return {
    ...
    'wrap_up_form': WrapUpForm(data={u'message': settings.DEFAULT_WRAP_UP_MESSAGE.format(name=customer.given_name.encode('utf-8'))}),
}

and the traceback I get is this

ERROR    2014-07-24 14:48:26,540 exception_handlers.py:65] 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/rony/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/home/rony/Documents/clone-attempt/personal-shopping/vendor/nacelle/core/dispatcher.py", line 24, in nacelle_dispatcher
    response = router.default_dispatcher(request, response)
  File "/home/rony/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/home/rony/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1065, in __call__
    return self.handler(request, *args, **kwargs)
  File "/home/rony/Documents/clone-attempt/personal-shopping/app/utils/decorators.py", line 43, in _arguments_wrapper
    return view_method(request, *args, **kwargs)
  File "/home/rony/Documents/clone-attempt/personal-shopping/app/utils/decorators.py", line 89, in _arguments_wrapper
    output = render_jinja2_template(template_name, context)
  File "/home/rony/Documents/clone-attempt/personal-shopping/vendor/nacelle/core/template/renderers.py", line 19, in render_jinja2_template
    return renderer.render_template(template_name, **context)
  File "/home/rony/google_appengine/lib/webapp2-2.5.2/webapp2_extras/jinja2.py", line 158, in render_template
    return self.environment.get_template(_filename).render(**context)
  File "/home/rony/google_appengine/lib/jinja2-2.6/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "templates/cms/appointments_form.html", line 2, in top-level template code
    {% import 'cms/macros.html' as cms_macros %}
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)

Do I need to add some sort of encoding to my templates?

Answer 1

It seems likely that customer.given_name is a byte string, rather than Unicode - so in calling encode on it, Python first needs to decode it to Unicode before it can then re-encode to UTF-8.

You should drop the encode call altogether.

Answer 2

As Daniel Roseman answered, I also suspect that customer.given_name is byte string; trying to encode on it cause Python try to decode it.

Another issue is that DEFAULT_WRAP_UP_MESSAGE is byte string literal.

str.format(unicode) has same issue.

---

Solution:

• remove .decode(..) part.
• Make a DEFAULT_WRAP_UP_MESSAGE unicode object instead of byte string.