Service Fabric Reliable Actors and I/O Operations

Question

On the Service Fabric Reliable Actors Introduction , documentation provided by Microsoft, it states that Actors should not, "block callers with unpredictable delays by issuing I/O operations."

I'm a bit unsure on how to interpret this.

Does this imply that I/O is ok so long as the latency of the request is predictable?

or

Does this imply that the best practice is that Actors should not make any I/O operations outside of Service Fabric? Like for example: to some REST API or to write to some sort of DB, data lake or event hub.

Answer 1

Technically, is a bit of both.

Because actors are Single Threaded, only a single operation can happen in the actor at same time.

The SF Actor uses the Ask approach, where every call expect an answer, the callers will make calls and keep waiting for answers, if the actor receives too many calls from clients, and this actor depends too much on external components, it will take too long to process each call and all other client calls will be enqueued and will probably fail at some point because they will wait for too long and timeout.

This wouldn't be a big issue for actors using the Tell approach, like Akka, because it does not wait for an answer, it just send the message to the mailbox and receive a message back with the answer(when applicable). but the latency between request and response will still be an issue because too many messages are pending to process by a single actor. On the other hand, could increase the complexity if one command fail, and there are 2 or 3 sequence of events that triggered before you know the answer for the first (not the scope here, but you relate this happening to the example below).

Regarding the second point, the main idea of an Actor is to be self contained, if it depends too much on external dependencies, maybe you should rethink the design and evaluate if the actor is actually the best design for the problem.

Self contained actors are scalable, they don't depend on external state manager to manage his own state, they won't depend on other actor to accomplish their tasks, they can scale independently of each other.

Example:

Actor1(of ActorTypeA) depends on Actor2 (of ActorTypeB) to execute an operation.

To make it more human friendly let's say:

• ActorTypeA is an Ecommerce Checkout Cart
• ActorTypeB is a Stock Management
• Actor1 is the Cart for user 1
• Actor2 is the Stock for product A

Whenever a client(user) interact with his checkout cart, adding or removing products, it will send add and remove commands to the Actor1 to manage his own cart. In this scenario the dependency is one to one, when another user navigate to the website, another actor will be created for him to manage his own cart. In both cases, they will have their own actors.

Let's now say, Whenever a product is placed in the cart, it will be reserved in stock to avoid double selling the same product.

In this case, both actor will try to reserve products in Actor2, and because of single threaded nature of Actors, only the first one will succeed and the second will wait the first to complete and fail if the product is not in stock anymore. Also, the second user, won't be able to add or remove any products to his cart, because the first operation is waiting to complete. Now increase these numbers for thousands and see how the problem evolves quickly and the scalability fails.

This is just a small and simple example, so, the second point is not just for External Dependencies, it also applies to internal ones, every operation outside the actor reduces the scalability of it.

Said that, you should avoid external (outside the actor) dependencies as much as possible , but is not a crime if it is needed, but you will reduce the scalability when an external dependency is limiting it to scale independently.

This other SO question I've answered might also be interesting to you.