Calling database object from different python modules

Question

I'm using MySQLdb module in Python 2.7 with a MariaDB database. I want to know how would be the proper way of using a database across multiple modules. I have this example in which I try to insert elements from another function into the database. I'm getting OperationalError: (1046, 'No database selected')

Edit: Here is the corrected code ( 1 , 2 ). Thanks. Know I would like a working example like this one with dependency injection.

This is the module in which I want to handle the database.

# test.py

import MySQLdb
import traceback
import test2 as insert_module
from warnings import filterwarnings

# Ignore MySQL `table already exists` warnings
filterwarnings ('ignore', category = MySQLdb.Warning)

# Create database object to keep it open as global variable
class  MyDB (object):
    connection = None
    cursor = None

    # Open database connection
    def  connect (self):
        self.connection = MySQLdb.connect ()
        self.cursor = self.connection.cursor ()

    def  execute (self, sql, *args):
        try:
            self.cursor.execute (sql, *args)
            self.connection.commit ()
        except  MySQLdb.Error, e:
            print (traceback.format_exc())
            self.connection.rollback ()

    def  query (self, sql, *args):
        try:
            self.execute (sql, *args)
        # Reopen database connection
        except  ( AttributeError, MySQLdb.OperationalError ):
            self.connect ()
            self.execute (sql, *args)

    def  disconnect (self):
        self.connection.close ()

db = MyDB ()

def  createDatabase ():
    db.query ("CREATE DATABASE IF NOT EXISTS Product")
    db.query ("USE Product")
    sql = """CREATE TABLE IF NOT EXISTS `ProductColor` (
                `ProductID` VARCHAR(20) PRIMARY KEY,
                `ColorID` VARCHAR(20)
            )"""
    db.query (sql)

def  insertProductColor (*args):
    sql = """INSERT INTO `ProductColor` VALUE (%s, %s)"""
    db.query (sql, args)

def  main ():
    createDatabase ()
    insert_module.processListOfProductColors ()

if __name__ == "__main__":
    main ()
    db.disconnect ()

This is the module from which I want to insert the products.

#test2.py

import test as db_module

def  processListOfProductColors ():
    db_module.insertProductColor ("cup", "blue")

Answer 1

Mostly unrelated but:

# Create database object to keep it open as global variable
class  MyDB (object):
    connection = None
    cursor = None

Those class attributes are useless - define them as instance attributes instead:

    def __init__(self):
        self.connection = None
        self.cursor = None

and while we're at it: DO NOT reuse the same cursor for all your queries. Else you're in for trouble when trying to execute another query while iterating over the results of a first one (and let's not talk multithreading...)

    # Open database connection
    def  connect (self):
        self.connection = MySQLdb.connect ()

You should certainly either check if you already have an opened connection, and if yes either close it first or just skip the MySQLdb.connect() call.

Also and FWIW, you have to pass connection infos (username, password, database name etc) to the MySQLdb.connect() call. Your OperationalError comes from not passing a database name here.

        self.cursor = self.connection.cursor ()

cf above : this will only get you troubles. Use a new cursor each time, that's the intended use of the dbapi.

    def  execute (self, sql, *args):
        try:
            self.cursor.execute (sql, *args)

cf above

            self.connection.commit ()

You should give the caller a way to NOT commit after each call - he may want to execute more than one statement in a single transaction (actually that's part of the point of having transactions...)

        except  MySQLdb.Error, e:

you want:

        except  MySQLdb.Error as e:

            print (traceback.format_exc())

Setup a logger (python's logging module) and call logger.exception(...) instead.

            self.connection.rollback ()

and ? Pretend it's ok ??? You want to re-raise the exception here obviously.

    def  query (self, sql, *args):
        try:
            self.execute (sql, *args)

Are you sure you want to commit transactions for read queries ???

        # Reopen database connection
        except  ( AttributeError, MySQLdb.OperationalError ):
            self.connect ()
            self.execute (sql, *args)

Err... If the point was to lazily handle the connection then you failed. Look at python's support for computed attributes instead ( property etc).

    def  disconnect (self):
        self.connection.close ()

More generally: trying to wrap the connection part in something that will gracefully (and eventually lazily) handle connection issues might be a good idea (but it's not that easy either...), but by all mean let the caller handle the cursor and transaction manually. A solution here could be to make execute() and query() context managers that close the cursor and commit (or rollback) the transaction. Also remember that a bad exception handling is worst than no exception handling...

Answer 2

ideally, you should pass your DB object through dependency injection into the insert_module - since it doesn't look like you've set up your code like that, you could ( as a quick hack , not the best solution) do:

insert_module.processListOfProductColors (self.db)

and then inside processListOfProductColors(db) use db.insertProductColor - as opposed to db_module.insert...

again, this is not the best solution without some refactoring, from what i can tell

update

here is a general example for dependency injection, which you can adapt for your own use:

# note that each class can be in a different module (file), 
# just make sure you import 

class db_class(object):
    def __init__(self):
        self.db = create_db()

    @staticmethod
    def create_db():
        db = # create the db
        return db

class main(object):
    db = db_class()
    first_class = first_class(db) # dependency injection part 1
    first_class.whatever()
    # you can pass db to however many classes you want now 

class first_class(object):
    def __init__(self, db):
        self._db = db # dependency injection part 2

    def whatever(self):
        # do whatever with self._db