Is there a way to check if a Django management command is running?

Question

The views rely on Redis to be populated. Redis is populated from a management command ran every 10 minutes. This management command deletes all existing keys and re-adds them with new data. How could I determine if the management command is running from a django view?

Right now I'm having the management command write to an external file and have a view read that file on each request. If the database is refreshing via the management command I hold up the view until it finishes (polling style).

Answer 1

Django does not provide a pre-packaged way to check whether an administration command is running. This being said, you should never write code that explicitly blocks a view while waiting for some result. You can easily use up all threads and processes that the server that runs your application has made available to your application. Your users will have a poor experience on your site, even those that don't do anything that has to do with the problem you're trying to solve here.

What I'm getting from your description is that you want users to get reasonably fresh results. For something like this I would use a solution based on versioning the data. It would go like this:

1. Declare a Redis-backed cache in your settings.py file that will contain the data populated by the command and read by the view. Make sure the TIMEOUT of the cache is set to NONE .

2. A current version number is recorded with the key CURRENT_VERSION . This key is itself unversioned.

3. When the command refreshes the data in the cache, it stores it in keys with version set to CURRENT_VERSION + 1 . You'll have something like:

    current_version = cache.get(CURRENT_VERSION) # Record the new data. for ...: cache.set(some_key, some_value, version=current_version + 1) 

4. Django's cache system does not readily allow getting a set of keys that correspond to a specific criterion. However, your view will want to obtain all keys that belong to a specific version. This information can be recorded as:

    cache.set(ALL_RECORDS, [... list of keys set in the loop above ...], version=current_version + 1) 

   Where ALL_RECORDS is a key value that is guaranteed not to clash with CURRENT_VERSION or any of the keys set for the individual records.

5. Once the command is done, it atomically increases the value of CURRENT_VERSION :

    cache.incr(CURRENT_VERSION) 

   The documentation on the Redis backend states that if you perform an increment on appropriate values (that's left vague but integers would seem appropriate) then Redis will perform the increment atomically.

6. The command should also clean up old versions form the cache. One method to ensure that old data does not stay in the cache is to set expiration times on the keys when you set their values. Your command refreshing the cache runs every 10 minutes. So you set keys to expire after 15 minutes. But suppose that a problem prevents multiple runs of the command to run. What then? Your data will be expired and removed from the cache, and the view will run with an empty data set. If this is okay for your situation, then I guess you could set the timeout parameter every time you do cache.set , except for CURRENT_VERSION which should never expire.

   If you are not okay with your view running with an empty data set (which seems more probable to me), then you have to write code in your command to seek old versions and remove them explicitly.

Your view accesses the cache by:

1. Reading the value of CURRENT_VERSION :

    current_version = cache.get(CURRENT_VERSION) 

2. Reading the list of records in the version it got:

    keys = cache.get(ALL_RECORDS, version=current_version) 

3. Processing the records:

    for key in keys: value = cache.get(key, version=current_version) 

The view should detect the case where the cache has not been initialized and fail gracefully. When deploying the application, care should be taken that the command has run at least once before the site can be accessed.

If the view starts working while the command is updating the cache, it does not matter. From the point of view of the command, the cache is just accessing the previous version. From the point of view of the view, the command is busy creating the next version but this is invisible to the view. The view does not have to block.