How to Create New Tables without Defined Classes in SQLAlchemy 1.2.7

Question

I am using SQLAlchemy's declarative API to build my models and tables for my application.

However, I'm unable to use Base.metadata.create_all(engine) command to build a new table in my database because of error NoReferencedTableError: Foreign key associated with column 'org_prediction.company_id' could not find table 'organization' with which to generate a foreign key to target column 'id'

The reason this is happening is because the class Organization I am trying to reference for a ForeignKey does not exist in my models.py file. So, when I call Base.metadata.create_all(engine) , SQLAlchemy chokes because class Organization is not present in Base...

I have tried reflecting my database, which will return the table for Organization which exists, but has no corresponding Class written for it in my models.py file. But when I reflect the DB tables and try to create my new dependent class on top of Organization, I get an error like: InvalidRequestError: Table 'user' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. InvalidRequestError: Table 'user' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. , which is caused because the reflected MetaData and my models.py class for User overlap, and SQLAlchemy isn't sure what to do.

But where do I apply extend_existing=True? I have no Table objects to pass this parameter into. How can I create my FK dependent table when my FK dependent table doesn't exist in my models.py file? Or how can I reflect my existing tables, but append or update only the classes which do not exist yet in the database?

models.py

from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer, primary_key=True)
    username = Column(String(50))

class OrgPrediction(Base):
    __tablename__ = 'org_prediction'
    id = Column(Integer, primary_key=True)
    company_id = Column(String(255), ForeignKey('organization.id'), nullable=True)
    prediction = Column(String(255)
Base.metadata.create_all(engine)

NoReferencedTableError: Foreign key associated with column 'org_prediction.company_id' could not find table 'organization' with which to generate a foreign key to target column 'id'

MYSQL:

mysql> desc organization;
+-----------------------+------------+------+-----+---------+-------+
| Field                 | Type       | Null | Key | Default | Extra |
+-----------------------+------------+------+-----+---------+-------+
| id                 | bigint(20) | YES  | MUL | NULL    |       |


mysql> show tables;
+-------------------------+
| Tables_in_database |
+-------------------------+
| organization            |
| user                    |
| user_email              |

In the case of Reflection the models.py file:

from sqlalchemy.ext.automap import automap_base
from sqlalchemy import create_engine

Base = automap_base()
Base.prepare(engine, reflect=True)

class User(Base):
    __tablename__ = 'user'
    id = Column(Integer, primary_key=True)
    username = Column(String(50))

class OrgPrediction(Base):
    __tablename__ = 'org_prediction'
    id = Column(Integer, primary_key=True)
    company_id = Column(String(255), ForeignKey('organization.id'), nullable=True)

Base.metadata.create_all(engine)

InvalidRequestError: Table 'user' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

Answer 1

There is a way to do this, and it involves using the Base.metadata.reflect(only=['table_name']) functionality prior to adding the Foreign-Key-dependency table. Final code that I tested:

Base = declarative_base()
class User(Base):
    __tablename__ = 'user'
    id = Column(Integer, primary_key=True)
    username = Column(String(50))

#see what tables exist in metadata right now
In [85]: Base.metadata.sorted_tables
Out[85]: [Table('user', MetaData(bind=None), Column('id', Integer(), table=<user>, primary_key=True, nullable=False), Column('username', String(length=50), table=<user>), schema=None)]

# this will add the organization table/class to the Base's Metadata. This also needs to happen before the Foreign Key reference in OrgPrediction class.
Base.metadata.reflect(bind=engine, only=['organization'])

# now this class' foreign key will reference organization.id which exists in Base metadata.
class OrgPrediction(Base):
    __tablename__ = 'org_prediction'
    id = Column(Integer, primary_key=True)
    company_id = Column(String(255), ForeignKey('organization.id'), nullable=True)
    prediction = Column(String(255))

# this should pass because the foreign key constraint is met.
Base.metadata.create_all(engine)