Google App Engine python inbound mail LookupError: unknown encoding

Question

I am receiving inbound email to my Google App Engine app, using a "standard" inbound mail handler, following the examples in the docs .

It seems that a certain email message sent to the app (not by me) is causing the email api to throw a LookupError: unknown encoding exception. Please see the traceback.

As far as I can tell this exception is thrown before the app's mail handler is invoked, apparently making it impossible to catch and handle by the app's code. Is this really the case ?

It also seems that App Engine retries the failed message at approximately 40 minutes intervals, which continues to produce errors (and alerts...) Is there any way to abort this ?

Thanks for your help.

Traceback:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/webapp/mail_handlers.py", line 70, in post
    self.receive(mail.InboundEmailMessage(self.request.body))
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 939, in __init__
    self.update_from_mime_message(mime_message)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 1513, in update_from_mime_message
    super(InboundEmailMessage, self).update_from_mime_message(mime_message)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 1422, in update_from_mime_message
    super(EmailMessage, self).update_from_mime_message(mime_message)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 1291, in update_from_mime_message
    subject = _decode_and_join_header(mime_message['subject'], separator=u'')
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 597, in _decode_and_join_header
    for s, charset in email.header.decode_header(header))
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/mail.py", line 597, in 
    for s, charset in email.header.decode_header(header))
LookupError: unknown encoding: iso-8859-8-i

Answer 1

The error is happening when the inbound mail handler's post method is called.

  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/webapp/mail_handlers.py", line 70, in post
    self.receive(mail.InboundEmailMessage(self.request.body))

The simplest solution is to override the post method in your own handler to trap the error:

import logging
from google.appengine.ext.webapp.mail_handlers import InboundMailHandler


class MyInboundMailHandler(InboundMailHandler):

    def post(self):
        try:
            super(MyInboundMailHandler, self).post()
        except LookupError as ex:
            logging.warning('Could not process message because %s.', ex)

    def receive(self, mail_message):
        # Process message

If you don't want to lose the message, you could create and register a custom iso-8859-8-i codec. This doesn't seem to be a well-documented process, but these questions provide some hints:

How do I properly create custom text codecs?

Custom Python Charmap Codec

how do I write a custom encoding in python to clean up my data?

And the standard library's iso-8859-8 encoding provides a good template.