Introduction to Distributed Publish-Subscribe in Akka.NET

Decentralized Message Brokers in Akka.Cluster

Today we’ll talk about one of the most common communication patterns, which is publish/subscribe, and how to perform it in a distributed environments using Akka.NET. In case if you want to save yourself some time implementing it, its already available as part of a bigger package called Akka.Cluster.Tools. To install it, you can simply get the latest prerelease version from NuGet:

install-package Akka.Cluster.Tools -pre

Below, we’ll cover the abilities and limitations that come with the Akka.Cluster.Tools.DistributedPubSub implementation.

When EventStream isn’t enough

An idea of distributed publish subscribe came from slightly different design choices than those of the default Akka pub/sub model (EventStream).

EventStream has been invented to work with actor systems from the ground up, even in systems that are not inherently distributed. It’s extremely simple, lightweight, and fast. Dead letters - a place where every undeliverable message lands by default - is implemented on top of it.

However, EventStream comes with some limitations. It works only locally in the scope of the current actor system. This means that you cannot publish/subscribe actors from other nodes and publish messages in a global address space without worrying on which node they live. These limitations were motivation for a separate, cluster-specific publish-subscribe feature.

Event stream publish/subscribe

Technical Overview of Akka.Cluster.Sharding in Akka.NET

How Akka.Cluster.Sharding Allocates Shards, Rebalances, and More

In our previous post about using Akka.Cluster.Sharding we looked at the module from an end-user’s perspective. Today we’ll provide a little more insights into how this plugin works internally.

Cluster sharding depends on several types of actors:

  • Coordinator - one per entity type for an entire cluster.
  • Shard region - one per entity type per each cluster node, where sharding should be enabled.
  • Shard - there can be many on each shard region, and they can move between shard regions located on different nodes.
  • Entity - actors defined by the end-user. There can be many on each shard, but they are always bound to a specific shard.

All of them can be visualized using diagram below:

Conceptual image of cluster shard internal actors located across the cluster nodes

If you look at actor paths of the entities you’ve created you’ll see that they reflect the structure of that hierarchy. They follow the pattern /user/sharding/<typeName>/<shardId>/<entityId>. Given that, it’s easy to infer that /user/sharding/<typeName> is path to a shard region while subsequent path segments are responsible for shard actor and entity actor.

Introduction to Akka.Cluster.Sharding in Akka.NET

Distributing State Evenly and Automatically with Self-Managing Actors

In this post, we’ll discuss one of the Akka.NET plugins, Akka.Cluster.Sharding and how it gives us easier, higher level abstractions to work with actors (in this case also referred to as entities). Cluster.Sharding gives us:

  • By using a logical keys (in form of ShardId/EntityId pairs) it’s able to route our messages to the right entity without need to explicitly determine on which node in the cluster it lives.
  • It automatically manages the process of actor creation, so you don’t have to do it explicitly. When a message is addressed to an entity that didn’t exist, an actor for that entity will be created automatically.
  • It’s able to rebalance actors across cluster as it grows or shrinks, to ensure its optimal usage.

Given all of those traits, let’s see how to utilize cluster sharding in a standard Akka.NET application.