在Python中管理redis连接

Question

I have gone through multiple questions on this topic and I'm still confused.I'm trying to find the optimal way to share a database connection between multiple Python modules. I have seen the Borg pattern as well but can't see how it would be useful when we need to share a database connection.

Here's what I have : I'm using Redis as my database and would like to share a redis connection between multiple modules.

Option 1: As suggested by Jared below, I could just create a redis connection and pass it everywhere.

class Crawler:
    def __init__(self, url_store):
        self._url_store = url_store


class UrlStore:
    def __init__(self, redis_client):
        self._redis_client = redis_client


if __name__ == '__main__':
    redis_client = redis.StrictRedis(host='localhost',
                                     port=6379,
                                     db=0,
                                     decode_responses=True)

    url_store = UrlStore(redis_client)
    c = Crawler(url_store)

Option 2: Wrap the redis connection in a class.

  class RedisConnection:
    client = redis.StrictRedis(host='localhost',
                               port=6379,
                               db=0,
                               decode_responses=True)


class Crawler:
    def __init__(self, url_store):
        self._url_store = url_store


class UrlStore:
    def __init__(self):
        self._redis_client = RedisConnection.client


if __name__ == '__main__':

    url_store = UrlStore()
    c = Crawler(url_store)

With Option 1, there's no need to worry about singleton etc , but I think we are leaking the database information everywhere.

Option 2 just doesn't feel elegant.

I think there is a bigger design question here ..which is ..should a database connection be "injected" wherever its needed ...or ..should it be accessed from a global/singleton/ shared state as needed ?

Answer 1

Just an Idea, Create a database instance pool. Say three/five. And have a dbPool dictionary

     dbPool = {client _inst1 = 0, client _inst2 = 0, client _inst3 = 0}

key -> db instance, value -> 0/1 (0- dbconn free, 1 dbconn inuse)

And Whenever you needs a instance have a small thread safe fn:

def getThreadSafeConnection(self) :
    for dbConn, value in self.dbPool.items() :
        if value == 0 :
            self.lock() # lock
            self.dbPool[dbConn] = 1 # db cursor in use 
            self.unlock()
            return dbConn
    return createNewConn() // if no dbConn is free

Once your query executed reset the dbInstance Flag

            self.lock()
            self.dbPool[dbConn] = 0 # db cursor is free
            self.unlock()

This will ensures proper query execution. If you feel more db hit may occur, then increase your dbPool size.