简体   繁体   English

"在 Alembic 迁移中使用 SQLAlchemy ORM:我该怎么做?"

[英]Using the SQLAlchemy ORM inside an Alembic migration: how do I?

I currently have a column that contains HTML markup.我目前有一列包含 HTML 标记。 Inside that markup, there is a timestamp that I want to store in a new column (so I can query against it).在该标记中,有一个我想存储在新列中的时间戳(以便我可以查询它)。 My idea was to do the following in a single migration:我的想法是在一次迁移中执行以下操作:

  1. Create a new, nullable column for the data为数据创建一个可以为空的新列<\/li>
  2. Use the ORM to pull back the HTML I need to parse使用 ORM 拉回我需要解析的 HTML<\/li>
  3. For each row对于每一行
    1. parse the HTML to pull out the timestamp解析 HTML 以提取时间戳<\/li>
    2. update the ORM object更新 ORM 对象<\/li><\/ol><\/li><\/ol>

      But when I try to run my migration, it appears to be stuck in an infinite loop.但是当我尝试运行迁移时,它似乎陷入了无限循环。 Here's what I've got so far:这是我到目前为止所得到的:

       def _extract_publication_date(html): root = html5lib.parse(html, treebuilder='lxml', namespaceHTMLElements=False) publication_date_string = root.xpath("\/\/a\/@data-datetime")[0] return parse_date(publication_date) def _update_tip(tip): tip.publication_date = _extract_publication_date(tip.rendered_html) tip.save() def upgrade(): op.add_column('tip', sa.Column('publication_date', sa.DateTime(timezone=True))) tips = Tip.query.all() map(tips, _update_tip) def downgrade(): op.drop_column('tip', 'publication_date')<\/code><\/pre>"

After a bit of experimentation using @velochy's answer, I settled on something like the following pattern for using SqlAlchemy inside Alembic.在使用@velochy 的答案进行了一些实验之后,我确定了类似以下模式的东西,用于在 Alembic 中使用 SqlAlchemy。 This worked great for me and could probably serve as a general solution for the OP's question:这对我很有用,可能可以作为 OP 问题的一般解决方案:

from sqlalchemy.orm.session import Session
from alembic import op

# Copy the model definitions into the migration script if
# you want the migration script to be robust against later
# changes to the models. Also, if your migration includes
# deleting an existing column that you want to access as 
# part of the migration, then you'll want to leave that 
# column defined in the model copies here.
class Model1(Base): ...
class Model2(Base): ...

def upgrade():
    # Attach a sqlalchemy Session to the env connection
    session = Session(bind=op.get_bind())

    # Perform arbitrarily-complex ORM logic
    instance1 = Model1(foo='bar')
    instance2 = Model2(monkey='banana')

    # Add models to Session so they're tracked
    session.add(instance1)
    session.add(instance2)

    # Apply a transform to existing data
    m1s = session.query(Model1).all()
    for m1 in m1s:
        m1.foo = transform(m1.foo)
    session.commit()

def downgrade():
    # Attach a sqlalchemy Session to the env connection
    session = Session(bind=op.get_bind())

    # Perform ORM logic in downgrade (e.g. clear tables)
    session.query(Model2).delete()
    session.query(Model1).delete()

    # Revert transform of existing data
    m1s = session.query(Model1).all()
    for m1 in m1s:
        m1.foo = un_transform(m1.foo)
    session.commit()

This approach appears to handle transactions properly.这种方法似乎可以正确处理事务。 Frequently while working on this, I would generate DB exceptions and they would roll things back as expected.经常在处理此问题时,我会生成数据库异常,并且它们会按预期回滚。

What worked for me is to get a session by doing the following:对我有用的是通过执行以下操作来获得会话:

connection = op.get_bind()
Session = sa.orm.sessionmaker()
session = Session(bind=connection)

You can use the automap extension<\/a> to automatically create ORM models of your database as they exist during the time of the migration, without copying them to the code:您可以使用automap 扩展<\/a>在迁移期间自动创建数据库的 ORM 模型,而无需将它们复制到代码中:

import sqlalchemy as sa
from alembic import op
from sqlalchemy.ext.automap import automap_base

Base = automap_base()

def upgrade():
    # Add the new column
    op.add_column('tip', sa.Column('publication_date', sa.DateTime(timezone=True)))

    # Reflect ORM models from the database
    # Note that this needs to be done *after* all needed schema migrations.
    bind = op.get_bind()
    Base.prepare(autoload_with=bind)  # SQLAlchemy 1.4 and later
    # Base.prepare(bind, reflect=True)  # SQLAlchemy before version 1.4
    Tip = Base.classes.tip  # "tip" is the table name

    # Query/modify the data as it exists during the time of the migration
    session = Session(bind=bind)
    tips = session.query(Tip).all()
    for tip in tips:
        # arbitrary update logic
        ...


def downgrade():
    op.drop_column('tip', 'publication_date')

Continue from the comments, you can try something like this:从评论继续,你可以尝试这样的事情:

import sqlalchemy as sa


tip = sa.sql.table(
    'tip',
    sa.sql.column('id', sa.Integer),
    sa.sql.column('publication_date', sa.DateTime(timezone=True)),
)


def upgrade():
    mappings = [
        (x.id, _extract_publication_date(x.rendered_html))
        for x in Tip.query
    ]

    op.add_column('tip', sa.Column('publication_date', sa.DateTime(timezone=True)))

    exp = sa.sql.case(value=tip.c.id, whens=(
        (op.inline_literal(id), op.inline_literal(publication_date))
        for id, publication_date in mappings.iteritems()
    ))

    op.execute(tip.update().values({'publication_date': exp}))


def downgrade():
    op.drop_column('tip', 'publication_date')

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 当我希望通过升级中的Session对象更改数据时,如何测试alembic迁移? - How do I test an alembic migration when I wish to alter the data via Session objects inside the upgrade? 如何在SQLAlchemy Alembic迁移中更新记录? - How to update a record in SQLAlchemy Alembic migration? 如何在 Flask-Migrate/Flask-SQLAlchemy 中正确编辑 Alembic 迁移脚本以添加或编辑列? - How do I properly edit Alembic migration scripts in Flask-Migrate/Flask-SQLAlchemy to add or edit columns? Alembic-SQLAlchemy初始迁移 - Alembic - sqlalchemy initial migration 如何在 Alembic 迁移 (Postgres) 中使用现有的 sqlalchemy 枚举 - How to use an existing sqlalchemy Enum in an Alembic migration (Postgres) Alembic SQLAlchemy迁移助手不跟踪现有表 - Alembic sqlalchemy migration assistant not tracking existing tables 如何在 SQLAlchemy ORM 中过滤给定字符串长度的列? - How do I filter a column for a given string length in SQLAlchemy ORM? 如何在 SQLAlchemy ORM 中创建跨不同模式的关系? - How do I create relationships across different schemas in SQLAlchemy ORM? 如何处理SQLAlchemy ORM中带有保留字符的数据库列? - How do I handle database columns with reserved characters in SQLAlchemy ORM? 如何在查询之前设置SQLAlchemy ORM类的属性? - How do I set the attributes of a SQLAlchemy ORM class before query?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM