Flask-Migrate/Alembic not detecting models using base class and multi-file structure

Question

This question has been asked a million times but none of the solutions seem to be working for me. I should note I've dealt with tangential issues quite often in other projects

Currently I'm using flask-sqlalchemy, flask-migrate, and postgresql.

File structure:

├── app
│   ├── __init__.py
│   ├── main
│   │   ├── __init__.py
│   │   ├── routes.py
│   │   └── users.py
│   ├── models
│   │   ├── annotations.py
│   │   ├── __init__.py
│   │   ├── mixins.py
│   │   └── users.py
├── config.py
├── docker-compose.yml
├── Dockerfile
├── icc2.py           <-- the app.py
├── migrations

app/__init__.py

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_cors import CORS
from elasticsearch import Elasticsearch
from flask_migrate import Migrate

from config import Config

db = SQLAlchemy()
migrate = Migrate()

def create_app(config_class=Config):
    app = Flask(__name__)
    app.config.from_object(config_class)

    db.init_app(app)
    migrate.init_app(app, db)

    app.es = Elasticsearch([app.config['ELASTICSEARCH_URL']]) \
        if app.config['ELASTICSEARCH_URL'] else None

    from app.main import bp as main_bp
    app.register_blueprint(main_bp, url_prefix='/_api')

    CORS(app, resources={r"/_api/*": {"origins": "*"}})
    return app

icc2.py

from app import create_app, db
from app.models import classes

app = create_app()

@app.shell_context_processor
def make_shell_context():
    print(db)
    return dict(db=db, **classes)

app/models/mixins.py

from app import db

from sqlalchemy.ext.declarative import declared_attr, as_declarative

@as_declarative()
class Base(db.Model):
    """This Base class does nothing. It is here in case I need to expand
    implement something later. I feel like it's a good early practice.

    Attributes
    ----------
    id : int
        The basic primary key id number of any class.

    Notes
    -----
    The __tablename__ is automatically set to the class name lower-cased.
    There's no need to mess around with underscores, that just confuses the
    issue and makes programmatically referencing the table more difficult.
    """
    __abstract__ = True
    id = db.Column(db.Integer, primary_key=True)

    @declared_attr
    def __tablename__(cls):
        return cls.__name__.lower()

app/models/users.py ( annotations.py is similar)

import time

from datetime import datetime

from app import db
from app.models.mixins import Base

class User(Base):
    auth0id = db.Column(db.String(64), index=True)
    last_seen = db.Column(db.DateTime, default=datetime.utcnow)

    def __repr__(self):
        return f"<User {self.displayname}>"

    def __str__(self):
        return self.displayname

app/models/__init__.py

import pkgutil
import os
import importlib
from .mixins import Base

pkg_dir = os.path.dirname(__file__)

for (module_loader, name, ispkg) in pkgutil.iter_modules([pkg_dir]):
    importlib.import_module('.' + name, __package__)

classes = {cls.__name__: cls for cls in Base.__subclasses__()}

The iteration comes from a stackoverflow snippet so that I can get the models in a class to expose to the flask shell namespace.

I perhaps don't need to put these two classes in two separate files, but my last project ended up with 30 or so models and so organization demanded a bit of a split, so it's just a practice I've developed.

It is my understanding that alembic needs to see the metadata for the objects before it can generate the models, but at the point in app/__init__.py where migrate is instantiated the db does not have an engine yet. In fact, just to test how the engine is created, I added 3 print statements to icc2.py to print the db and see if it has an engine at that point, like so:

from app import create_app, db
from app.models import classes

app = create_app()
print(db)
@app.shell_context_processor
def make_shell_context():
    print(db)
    return dict(db=db, **classes)
print(db)

The only print call that didn't show <SQLAlchemy engine=None> was within the make_shell_context() function.

Finally, the output of flask migrate "Initial migration":

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.env] No changes in schema detected.

So how do I expose my metadata to flask migrate?

Answer 1

Importing * from the model modules inside alembic's env.py worked for me. Organizing the all the model modules and alongside the base module as * imports in a __init__.py and importing from that aggregation module in env.py also works.