How do I handle “out of range” warnings for timezone-aware datetime objects on MySQL with and without SQLAlchemy

Question

I receive a warning from Mysql-Python when inserting a timezone-aware datetime object into a DateTime column on MySQL:

test_mysql.py:13: Warning: Out of range value for column 'created_at' at row 1
  cur.execute("INSERT INTO test (created_at) VALUES (%s)", now)

The code for the test looks like this:

import MySQLdb
from datetime import datetime
from pytz import utc

conn = MySQLdb.connect(...)  # connect
cur = conn.cursor()
now = datetime.utcnow()
cur.execute("CREATE TABLE test (created_at DATETIME)")
print("Test 1")
cur.execute("INSERT INTO test (created_at) VALUES (%s)", now)
now = utc.localize(now)
print("Test 2")
cur.execute("INSERT INTO test (created_at) VALUES (%s)", now)
print("Test 3")
now = now.replace(tzinfo=None)
assert now.tzinfo is None
cur.execute("INSERT INTO test (created_at) VALUES (%s)", now)
print("Tests done")
cur.execute("DROP TABLE test")

With the full output being:

Test 1
Test 2
test_mysql.py:13: Warning: Out of range value for column 'created_at' at row 1
  cur.execute("INSERT INTO test (created_at) VALUES (%s)", now)
Test 3
Tests done

I am using SQLAlchemy in my original application but since this is a problem below SQLAlchemy I am interested in solutions for both with SA and without.

Answer 1

Without SQLAlchemy the answer is basically already in the question: Make it naive but in such a way that you always store the same (read: UTC) timezone:

if now.tzinfo is not None:
    now = now.astimezone(utc).replace(tzinfo=None)
return now

This will make sure that UTC is always stored (but if passing naive make sure that it is always naive ! And when pulling the item out of the database, make sure to make it UTC-aware again:

if created_at.tzinfo is None:
    created_at = utc.localize(created_at)
return created_at

Since MySQL is not capable of tz-aware objects, the check could be removed (it should always be None ).

For SQLAlchemy, the same approach can be used (but with the extension of letting SA do it automatically). For this we use the TypeDecorator :

class TZDateTime(TypeDecorator):
    """
    Coerces a tz-aware datetime object into a naive utc datetime object to be
    stored in the database. If already naive, will keep it.

    On return of the data will restore it as an aware object by assuming it
    is UTC.

    Use this instead of the standard :class:`sqlalchemy.types.DateTime`.
    """

    impl = DateTime

    def process_bind_param(self, value, dialect):
        if value.tzinfo is not None:
            value = value.astimezone(utc).replace(tzinfo=None)
        return value

    def process_result_value(self, value, dialect):
        if value.tzinfo is None:
            value = utc.localize(value)
        return value

By this it is made sure that the data handled is always tz-aware on UTC as long as only naive values are always generated to UTC: datetime.datetime.utcnow() for example.

When defining a column, use that instead of the regular type:

class Test(Base):
    __tablename__ = 'test'
    created_at = Column(TZDateTime)