使用ClusterSharding在Akka群集中存储实体ID

Question

I have an akka.net cluster where I intend to use ClusterSharding to shard entities (actors) across the nodes in the cluster. Sitting above the cluster is a layer that will be a standard REST style API offering endpoints to create and retrieve the entities. I would also like to be able to return, through this API, the IDs of all the entities currently in the cluster (as you would normally in a REST style API, for example via a GET on /api/entity) - these IDs would be a property of the entity or could be the full ID used by the ClusterSharding module when directing messages.

Is there an easy way to retrieve all these IDs from the cluster? Or should I create a separate 'caching' actor that stores the IDs as and when the entities get created / recovered (in the case of a persistent actor) that the REST API layer can call directly? Or should the API layer query the cluster for the IDs each and every time it gets called (by broadcasting a message to all the entities and aggregating the responses)? This last one seems like it could be a bit wasteful, particularly in times when the list of entities is relatively static.

Or am I missing something and need to design this differently?

Thanks for any help you can give.

Answer 1

If you're interested only in entities that are currently alive, you can simply ask current shard region for its entity IDs:

var region = ClusterSharding.Get(system).ShardRegion(typeName);
var state = await region.Ask<CurrentShardRegionState>(GetShardRegionState.Instance);
foreach (var shard in state.Shards) 
    foreach(var entityId in shard.EntityIds)
        Console.WriteLine($"{typeName}/{shard.ShardId}/{entityId}");

This however will retrieve only information about entities living on a current cluster node, not everywhere. You could however aggregate results if necessary:

var cluster = Cluster.Get(system);
var region = ClusterSharding.Get(system).ShardRegion(typeName);
var members = cluster.State.Members;
var regions = members
    .Select(m => region.Path.ToStringWithAddress(m.Address))
    .Select(system.ActorSelection);

// at this point regions variable is a collection of ActorSelections poiting
// to regions on all known nodes. You can use Ask pattern on them directly

Keep in mind, that that kind of request could be quite huge.

While this one will allow you to retrieve informations about all EntityIDs on all nodes for a particular typeName, it still won't count inactive entities.

If you'd want to get IDs on all entities living on all nodes ever, you'd need to perform a query over a ReadJournal used by your sharding module. This requires two conditions to be met:

1. It works only if your cluster sharding extension is using persistent backend supporting Akka.Persistence.Query .
2. You need akka.cluster.sharding.remember-entities = on setting - it's turned off by default, and it will increase the cost of cluster sharding infrastructure once set on.

In case of SQL journals, it should look more or less like this:

var queries = PersistenceQuery.Get(actorSystem)
    .ReadJournalFor<SqlReadJournal>("path-to-sharding-journal");

var shardPrefix = "/sharding/" + typeName + "Shard/"
Source<string, NotUsed> entitySource = queries
    .AllPersistenceIds() 
    .Where(pid => pid.StartsWith(shardPrefix)) // get all shard IDs ever known
    .ConcatMany(shardId => queries.CurrentEventsByPersistenceId(shardId)) // get stream of persisted events for each shard (you may need to stretch the query limit here)
    .Collect(env => env.Event as Shard.EntityStarted) // filter out EntityStarted events
    .Select(e => e.EntityId); // extract entity ID

using (var mat = system.Materializer())
{
    // materialize source into a list
    var entityIds = await entitySource
        .RunAggregate(ImmutableList<string>.Emtpy, (acc, id) => acc.Add(id), mat);
}

Keep in mind that this kind of query may also be pretty expensive.