Form Akka.NET Clusters Dynamically with Akka.Management and Akka.Discovery

A safer, superior choice to using seed nodes with Akka.Cluster

15 minutes to read

Akka.Cluster is a very powerful piece of software aimed at helping .NET developers build highly available, low-latency, distributed software. At its core, Akka.Cluster is about building peer-to-peer networks—that’s what a “cluster” actually is: a peer-to-peer network that runs in a server-side environment controlled by a single operator.

What Clusters Need

This is a subject for another blog post, but what makes peer-to-peer networks a superior choice over client-server networks for high availability are the following:

  1. Horizontally scalable, because the “source of truth” is decentralized and distributed to the endpoints of the network (these are your actors running in each process) rather than centralized in a single location;
  2. Fault tolerant and resilient - having the source of truth decentralized and distributed also means that no single node in the network is so crucial that its disappearance is going to render the system unavailable; and
  3. Still supports inter-dependent services - you can still have multiple services with completely different purposes and code cooperating together inside a peer-to-peer network. This is what Akka.Cluster roles are for.

In order to build a peer-to-peer network, you need two primary ingredients:

  1. Topology-awareness - database-driven CRUD applications never need to do this. The load-balancer is aware of where the web servers are and the web servers are aware of where the database is, but that’s pretty much it. In a real peer-to-peer network, all applications need to know about each other and need to communicate with each other directly. These are what Akka.Remote (communication) and Akka.Cluster (topology) provide.
  2. Initial formation - there must be a strategy for processes to form a new peer-to-peer network or to join an existing one.

In this blog post, we’ll be discussing item number 2—how to make the formation and joining of Akka.Cluster networks more reliable,...

Introducing Incrementalist, an Incremental .NET Build Tool for Large Solutions

Reduced Akka.NET's average build time from ~1.25hrs to 15 minutes.

12 minutes to read

We blog a ton about Akka.NET here, but Petabridge really is a .NET open source company. Throughout our work on Akka.NET we create many other OSS tools in support of much more general purpose .NET use cases, such as the .NET Runtime Dashboards we touted on our YouTube channel a couple of weeks ago or, back in the day, tools like NBench1.

Today though we’re writing about a brand new tool we’ve been working on for the past several months: Incrementalist v1.0, a command-line tool that leverages git and Roslyn solution analysis to drastically reduce build times for large .NET solutions and monorepos.

Incrementalist - a Git-based incremental build and testing platform for .NET and .NET Core.

We’ve been using older versions of Incrementalist in production inside the Akka.NET build pipeline since 2019 - it cuts our average pull request build time down from about 1 hour and 15 minutes to ~15 minutes. Those older versions of Incrementalist just spat out the smallest possible build graphs as a .csv file - it was up to you to parse it and use the data accordingly.

Incrementalist v1.0 is a totally different animal: it runs the dotnet commands for you.

For example, from Akka.NET’s live build system:

dotnet incrementalist run --config .incrementalist/testsOnly.json -- test -c Release --no-build --framework net8.0 --logger:trx --results-directory TestResults 

This call:

  1. Invokes the run verb - which means we’re going to execute a dotnet command against all of the matching projects (C# or F#);
  2. Uses the

Retiring Akka.Persistence.SqlServer, Postgres, Sqlite

Akka.Persistence.Sql is the new flavor moving forward.

8 minutes to read

It was just about 10 years ago when we shipped Akka.NET v1.0.2, the release where we first introduced betas of some of our most popular Akka.Persistence plugins: Akka.Persistence.Postgres, Akka.Persistence.SqlServer, and Akka.Persistence.Sqlite.

All of these plugins were based off of a shared ADO.NET Akka.Persistence architecture called Akka.Persistence.Sql.Common and this architecture has served both us and our users / customers well over the past 10 years, somewhere to the tune of 1.6 million installations!

But, as of Akka.NET v1.5.40 these plugins are all deprecated in favor of Akka.Persistence.Sql - the successor plugin for all SQL RDBMS we introduced along with the release of Akka.NET v1.5.

Akka.Persistence.SqlServer, Postgres, and Sqlite have all been retired in favor of Akka.Persistence.Sql going forward

In the next sections we’ll explain our decision along with showing you our migration guide for moving off any of the affected Akka.Persistence.Sql.Common plugins and onto Akka.Persistence.Sql.

.NET Heisenbug Mystery Theater: How Did an Exception Escape its Catch Block?

A painful lesson on atomicity and the assignment of structs.

21 minutes to read

Over the past several months the Akka.NET team has had reports of the following Exception popping up unexpectedly throughout many of our plugins and end-user applications that use the Akka.Streams1 SelectAsync stage - such as Akka.Streams.Kafka and Akka.Persistence.Sql:

ArgumentException stemming from the cancellation of an Akka.NET stream without a cause

That error message seems simple enough - it comes from here inside GraphStage.cs:

[InternalApi] public void InternalOnDownstreamFinish(Exception cause) { try { if (cause == null) throw new ArgumentException("Cancellation cause must not be null", nameof(cause)); 

In Akka.Streams parlance, a stream gets cancelled when an unhandled Exception is thrown and that error should be propagated all the way down to this GraphStage.InternalOnDownstreamFinish method so we can log why the stream is being cancelled / terminated.

Here’s the mystery - this is the code that “threw” the Exception inside Akka.Persistence.Sql for instance:

.SelectAsync( JournalConfig.DaoConfig.Parallelism, async promisesAndRows => { try { await WriteJournalRows(promisesAndRows.Rows); foreach (var taskCompletionSource in promisesAndRows.Tcs) taskCompletionSource.TrySetResult(NotUsed.Instance); } catch (Exception e) { foreach (var taskCompletionSource in promisesAndRows.Tcs) taskCompletionSource.TrySetException(e); } return NotUsed.Instance; }) 
...

Introducing Akka.NET Bootcamp 2.0

Modern Akka.NET Best Practices in Free, Self-Paced Lessons

2 minutes to read

Akka.NET Bootcamp has a special place in my heart because it’s really “the thing” that launched Petabridge as a business 10 years ago.

However, the original Akka.NET Bootcamp should have really been replaced years ago as both Akka.NET and the .NET ecosystems had changed tremendously since it was originally written.

So on that long-overdue note, it’s my pleasure to announce that the first two units of Akka.NET Bootcamp 2.0 are now live and available for immediate consumption!

Akka.NET Bootcamp Logo

Here’s what’s new in Bootcamp 2.0:

10 Important Engineering Lessons Learned Over 10 Years of Petabridge

Petabridge turns 10 in January 2025!

1 minute to read

When we first started Petabridge in January 2015, I never imagined that we’d be at this for 2-3 years, let alone 10! But here we are. It’s been an amazing journey and our privilege to work with so many amazing customers and software developers on building some of the world’s most important software on top of Akka.NET.

I already published a small post on my personal website about the business side of running Petabridge over this span of time, but for our subscribers and Akka.NET users here at Petabridge, I thought it’d be more appropriate to share some of the engineering lessons we’ve learned over the course of 10 years from all of our experiences here.

Please enjoy “10 Most Important Engineering Lessons Learned from 10 Years of Petabridge”.

The Lessons

Akka.NET v1.6 Roadmap and Features

A performance-focused release

7 minutes to read

We’re kicking off 2025 with a fresh look at Akka.NET and what we have planned for our next big release: Akka.NET v1.6!

Akka.NET v1.6: a Performance-Focused Release

Akka.NET v1.6 - a performance-focused release

As it says in our promo image here - Akka.NET v1.6 is a performance-focused release. To that end, we’re addressing the following three major changes:

How to Start Learning Actor-Based Programming

Discovering and appreciating differences between Akka.NET actors and traditional Object-Oriented Programming

7 minutes to read

One of the most frequent pieces of feedback we get from developers who are new to Akka.NET is that the “learning curve” is high. I want to explore that today and why I think this isn’t actually true, but what developers are saying is that working with actors is unfamiliar, not difficult.

I’d also argue that really, really bad pre-existing habits many software developers have, such as frameworkism, become a hundred-fold more destructive when you introduce unfamiliar paradigms, such as stateful programming with actors, to the mix. You could substitute “stateful programming with actors” with “using NoSQL” or “going cloud native” and that sentence would be equally true.

Let’s assume you’ve already decided that learning actors is worth your time and now you want to know: “how can I begin to learn how to work with actors for my own purposes?” That’s what this post is about.

Why IActorRef.Tell Doesn't Return a Task

Nuances of actor-based programming in .NET

9 minutes to read

A really, really common question Akka.NET beginners have within the first couple of hours of looking at Akka.NET: “how is the IActorRef.Tell method asynchronous if it’s a void method? Shouldn’t it return a Task I can await on?”

Today I’m going to clarify Akka.NET’s API design and explain, in terms of Akka.NET’s behavior and benefits to users, why we don’t do this.

Even if you’re a seasoned Akka.NET user, you might find some value in this post.

Akka.NET Actors' Hidden Super Power: Behavior Switching

Making the Complex Understandable

13 minutes to read

We put together a new YouTube video on our Petabridge channel (which you should subscribe to) yesterday, “Akka.NET Actors’ Hidden Super Power: Switchable Behaviors” - all about one of my favorite features in Akka.NET: the Become method for swapping actor message-processing behavior dynamically at runtime.

In addition to the video, I wanted to expand on why behavior-switching is so powerful and how you can use it to transform inherently complex domain problems into something that is approachable, understandable, and expressable with a very small amount of purpose-built code.