如何在另一个进程的原子事务期间读取修改后的表？

Question

With Django I am doing a big processing with a background process that does a lot of updates in the database. I protect the whole big processing with an atomic transaction, thus if it fails, the database go back to its previous state. In order to follow in which step the processing is, I update a table dedicated for that. I would like to display in real time in which step the processing is on a web site (so, from another process). Unfortunatly, I cannot see steps changing in the dedicated table because the transaction is committed only at the end of the processing. To summary, I have someting like this:

For the background process:

with transaction.atomic():
    task.step = 'Step 1'
    task.save()

    ... do step 1 processing ...

    task.step = 'Step 2'
    task.save()

    ... do step 2 processing ...

    task.step = 'Finised'
    task.save()

On web site, I will have an Ajax view that will do something like that:

GetProcessingProgressView(View):
    def get(self, request, **kwargs):
        ... 
        task = Task.objects.get(....)
        data = { 'step' : task.step }
        return HttpResponse(json.dumps(data), content_type='application/json')

The only step displayed on the web site is 'Finished' (that is when the transaction is committed)

How can I follow my big processing steps from another process while the transaction is not committed yet? Is there a way to exclude a table from the atomic transaction? Actually, the only solution I see is to use another database for that, or use files instead of database: what would you do instead?

Answer 1

You would have to choose a communication mechanism that is outside the database, since almost everything in PostgreSQL is transactional.

As a workaround, you could use the dblink extension to open a second database session to the same database using a database function and update the progress tracking table this way.