How to disable DDL transaction in an alembic migration

Question

I am trying to run an alembic transaction. However, all migrations run in a transaction whenever transactions are supported (see Run alembic upgrade migrations in a transaction ). How do I disable transaction for a specific migration?

Answer 1

Alembic used to have just two modes of using transactions:

• One transaction for the whole migration command. If there are multiple versions to apply, then they all run in that single transaction.
• Use a separate transaction per migration step.

However, as of version 1.2.0 (released September 2019), you can now also switch to the AUTOCOMMIT transaction level by using the MigrationContext.autocommit_block() context manager . When in this transaction mode, each statement is committed immediately. Note that there are caveats to using this feature, see below.

By default a single transaction is used, but you can call context.configure() in your env.py script to set transaction_per_migration to true to use separate transactions.

The first and default option, to use a single transaction, is executed in the env.py file that Alembic generates for you, in the run_migrations_online() function in that file:

try:
    with context.begin_transaction():
        context.run_migrations()
finally:
    connection.close()

You could either just edit that file to remove the with context.begin_transaction(): context manager, or use the context.get_x_argument() feature to toggle transactions on the basis of a command-line switch:

try:
    # Python 3.7+
    from contextlib import nullcontext
except ImportError:
    # Earlier Python versions
    from contextlib import contextmanager
    @contextmanager
    def nullcontext():
        yield

# ...

def run_migrations_online():
    # ...
    if context.get_x_argument(as_dictionary=True).get('no-transaction', False):
        transaction_cm = nullcontext()
    else:
        transaction_cm = context.begin_transaction()
    try:
        with transaction_cm:
            context.run_migrations()
    finally:
        connection.close()

To disable a transaction per migration step or for specific operations, you can use the aforementioned autocommit_block() , which is intended to be used for DDL statements that the database requires to be run outside of a transaction context:

def upgrade():
    with op.get_context().autocommit_block():
        op.execute("ALTER TYPE mood ADD VALUE 'soso'")

The above example (taken from the documentation), uses the Operations.get_context() method to get access to the migration context. Within the context, all statements are executed directly, without running in a transaction.

The caveat is that any transaction currently in progress is committed first . If statements before and after such a block are connected and should not be executed without the others, then you want to avoid placing an autocommit_block() in between. You also probably want to set transaction_per_migration = true , and use autocommit_block() for entire migration steps. That way you can at least minimise issues with a migration step failing halfway through.

Before version 1.2.0, it was not easy to disable transactions per migration step . You'd have do disable transactions entirely (just don't use context.begin_transaction() in env.py ), then explicitly use a transaction per upgrade() or downgrade() step:

def run_migrations_online():
    # ...

    try:
        # no with context.begin_transaction() here
        context.run_migrations()
    finally:
        connection.close()

and in each migration step:

def upgrade():
    with context.begin_transaction():
        # ### commands auto generated by Alembic - please adjust! ###
        op.create_table(
            # ...
        )
        # etc.

Answer 2

This can be done using an autocommit block:

with op.get_context().autocommit_block():
  op.execute(...)

https://alembic.sqlalchemy.org/en/latest/api/runtime.html#alembic.runtime.migration.MigrationContext.autocommit_block

This special directive is intended to support the occasional database DDL or system operation that specifically has to be run outside of any kind of transaction block. The PostgreSQL database platform is the most common target for this style of operation, as many of its DDL operations must be run outside of transaction blocks, even though the database overall supports transactional DDL.

Note that there are some caveats:

Warning: As is necessary, the database transaction preceding the block is unconditionally committed. This means that the run of migrations preceding the operation will be committed, before the overall migration operation is complete. It is recommended that when an application includes migrations with “autocommit” blocks, that EnvironmentContext.transaction_per_migration be used so that the calling environment is tuned to expect short per-file migrations whether or not one of them has an autocommit block.