How to track requests and completion status in async systems?

Question

I am trying to explore solutions to track requests in Async systems. This is my use case:

• Currently in our system we have 2 services communicating asynchronously with Service 1 sending an operation to be performed against n items to Service 2 .
• Service 2 on completion of those operations sends response back to Service 1 .

Currently how tracking is handled is via a request tracker table in Service 1 's DB from which we keep removing entries as and when we get response from Service 2 .

I was thinking if there is a more generic open source solution which I can leverage to track pending operations and perform some action once all pending are completed.

Anyone had a similar use case before?

Answer 1

Look at temporal.io open source project which solves the problem of long-running requests as well as many service orchestration use cases. In your example, the request to service1 can be modeled as a single blocking function call independent of the duration of the request processing (can be months).