Python MySQL : Refresh data when changes on database

Question

I have a Python API running in a Docker container, that API uses data from a MySQL database, but always the most recent data. Currently, each time we handle a query on the API, we run a SELECT query on the db to get the last version of the data. The problem is that's absolutely not efficient in term of performances and ressources since the responses are getting bigger and bigger. I tried to set up something with a Docker volume and a scheduled script to update the volume daily but it's a real gaz factory and it's not fast enough.

My question is: Is there a way to create a connection that can update our local data by running a SELECT query each time new data are commited on the db?

Our db query is quite simple, it's just a SELECT field_1, field_2 FROM table WHERE field_3 = "value"

And we are using PyMySql to create the connection as follow:

cnx = pymysql.connect(
        host="address",
        user="user",
        password="password",
        db="db_name",
        charset="utf8mb4",
        cursorclass=pymysql.cursors.DictCursor,
    )

Thanks in advance for your help !

Answer 1

That specific statement would run faster if you had

INDEX(field_3, field_1, field_2)

If you have multiple values in the table, but want the 'latest', then do something like:

SELECT ...
    ORDER BY dt DESC
    LIMIT 1

(Provide specifics about the table if you want more details)