Django: A common/global variable that is constructed on server start and destroyed on server exit

Question

I'd like to use Django as a relay to another layer. When the user makes a request, a thrift client calls a function that goes to another software to retrieve the result.

I understand the potential thread-safety issues, which is why there will be an object pool of clients based on Python's multiprocessing, and every user request will pop one, use it, and put it back.

The object pool has to be initialized at the beginning of the program, and destroyed (hopefully cleanly) when the program exits (when the server is interrupted with SIGINT). The motivation to this question is to know how to do this in the correct way without hacks.

The only way I found so far is the following prototype with global variables, given that the following is the views file with function based views:

clients_pool = ClientsPool("127.0.0.1", 9090, size=64) # pool size

def get_data(request):
    global clients_pool
    c = clients_pool.pop()  # borrow a client from the pool
    res = c.get_data(request.body)  # call the remote function with thrift
    clients_pool.release(c)  # restore client to the pool
    return HttpResponse(res)


def get_something(request):
    global clients_pool
    c = clients_pool.pop()  # borrow a client from the pool
    res = c.get_something(request.body)  # call the remote function with thrift
    clients_pool.release(c)  # restore client to the pool
    return HttpResponse(res)

Is this the correct way to do this? Or does Django offer a better mechanism to do this?

Answer 1

I would make sure multiple goals are satisfied here. First make sure always only one instance of your pool manager is created. You could place it in your app's __init__.py . I don't know if this threadsafe. I doubt it.

Second you want to delegate the management of the pool to the application layer. Right now you are managing the pool in the view layer. You always want to make sure pool clients are released I suppose. This would lead to something like this:

# [app_name]/__init__.py

class Singleton(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]


class PoolManager(metaclass=Singleton):
    def __init__(self, *args, **kwargs)
        self.pool = ClientsPool(*args, **kwargs)


pool_manager = PoolManager(host='127.0.0.1', port=9090, size=64)

# views.py / consider moving the context manager to seperate file
from [app_name] import pool_manager


class PoolContextManager:
    def __init__(self):
        self.pool = pool_manager

    def __enter__(self):
        return self.pool.pop()

    def __exit__(self, type, value, traceback):
        self.pool.release(c)


def get_data(request):
    with PoolContextManager() as pm:
        res = pm.get_data(request.body)  
        return HttpResponse(res)


def get_something(request):
    with PoolContextManager() as pm:
        res = pm.get_something(request.body) 
        return HttpResponse(res)