Azure service fabric actor for long running tasks

Question

I am very new to Azure Service Fabric. From my research I am quite keen in utilizing it for a specific optimization problem scenario. However I do not find any detail information on MSDN to back my findings.

My requirement is that I have a web API that gets an input and dumped to a database. This input is used to run an optimization algorithm which typically takes around 3-5 minutes. There may be multiple requests submitted by users which will need to be processed eventually.

I currently think spawning up a new ASF reliable actors per optimization input is a good choise. However I am not clear on how ASF functions for such tasks that are long running and not instantaneous.

Further I am in doubt how my resource utilization within the cluster will be. My end goal is to be able to submit at least a predefined amount of optimization algorithm actors or stateless services when required in parallel.

Really appreciate your technical advice related to the concerns I have. Is it actors or stateless that I should consider for this scenario.

Answer 1

I think what's essential here is to do the calculations in the background. Service /Actor callers shouldn't be kept waiting while the optimization algorithm runs.

To accomplish this you'd need to take a command, save it somewhere (a Queue would do nicely), return a token to the caller. The token can be used for querying status/progress.

Doing this in an Actor requires:

• Using the StateManager to hold a queue, that holds jobs
• Registering a timer to process the work.
• Optionally calling another (extra) Actor to report on progress

Doing this in a Stateless service requires:

• An external queue (which is an external dependency, that impacts availability)
• Optionally an external progress store

Doing this in a Stateful service (additional choice) requires:

• using the StateManager to hold a ReliableQueue .
• periodically checking this queue for work from RunAsync .
• Optionally periodically store progress in the StateManager .

I suspect the service type that's best prepared for this scenario is the Stateful Service. Here's a Stateful Service example that queues and processes work.

Answer 2

I think a combination of stateful and stateless will be a good solution for your scenario. Use the stateful to queue the requests and use the stateless to process (algorithms) the requests. The SF cluster will create new instances of the stateless service to other nodes in the cluster if needed based on the load. The stateful service will be your durable store to store the requests(via reliable queue) and progress(reliable dictionaries). The stateful service maintains one primary replicas and two secondary replicas in the cluster.