sqlalchemy caching some queries

Question

I have this running on a live website. When a user logs in I query his profile to see how many "credits" he has available. Credits are purchased via paypal. If a person buys credits and the payment comes through, the query still shows 0 credits even though if I run the same query in phpmyadmin it brings the right result. If I restart the apache webserver and reload the page the right number of credits are being shown. Here's my mapper code which shows the number of credits each user has:

mapper( User, users_table, order_by = 'user.date_added DESC, user.id DESC', properties = {
    'userCreditsCount': column_property( 
        select( 
            [func.ifnull( func.sum( orders_table.c.quantity ), 0 )],
            orders_table.c.user_id == users_table.c.id
        ).where( and_( 
            orders_table.c.date_added > get_order_expire_limit(), # order must not be older than a month
            orders_table.c.status == STATUS_COMPLETED
        ) ).\
        label( 'userCreditsCount' ),
        deferred = True
    )
    # other properties....
} )

I'm using sqlalchemy with flask framework but not using their flask-sqlalchemy package (just pure sqlalchemy)

Here's how I initiate my database:

engine = create_engine( config.DATABASE_URI, pool_recycle = True )
metadata = MetaData()
db_session = scoped_session( sessionmaker( bind = engine, autoflush = True, autocommit = False ) )

I learned both python and sqlalchemy on this project so I may be missing things but this one is driving me nuts. Any ideas?

Answer 1

when you work with a Session, as soon as it starts working with a connection, it holds onto that connection until commit(), rollback() or close() is called. With the DBAPI, the connection to the database also remains in a transaction until the transaction is committed or rolled back.

In this case, when you've loaded data into your session, SQLAlchemy doesn't refresh the data until the transaction is ended (or if you explicitly expire some part of the data with expire()). This is the natural behavior to have, since due to transaction isolation , it's very likely that the current transaction cannot see changes that have occurred since that transaction started in any case.

So while using expire() or refresh() may or may not be part of how to get the latest data into your Session, really you need to end your transaction and start a new one to truly see what's been changed elsewhere since that transaction started. you should organize your application so that a particular Session() is ready to go when a new request comes in, but when that request completes, the Session() should be closed out, and a new one (or at least a new transaction) started up on the next request.

Answer 2

Please try to call refresh or expire on your object before accessing the field userCreditsCount :

user1 = session.query(User).get(1)
# ...
session.refresh(user1, ('userCreditsCount',))

This will make the query execute again (when refresh is called).

However, depending on the isolation mode your transaction uses, it might not resolve the problem, in which case you might need to commit/rollback the transaction (session) in order for the query to give you new result.

Answer 3

Lifespan of a Contextual Session

I'd make sure you're closing the session when you're done with it.

session = db_session()
try:
  return session.query(User).get(5)
finally:
  session.close()

Answer 4

set sessionmaker 's autocommit to True and see if that helps, according to documentation sessionmaker caches

the identity map pattern, and stores objects keyed to their primary key. However, it doesn't do any kind of query caching.

so in your code it would become:

sessionmaker(bind = engine, autoflush = True, autocommit = True)