Akka.NET vs. Kafka, RabbitMQ, and Other Messaging Systems

What's the difference between Akka.NET, Kafka, RabbitMQ, and other message-driven technologies? Can they work together?

A while back I created a thread on Twitter to attempt to explain the difference between Akka.NET and some other popular message-distribution and queuing technologies, such as Apache Kafka and RabbitMQ.

I’m going to cover that in some more detail in this post because it’s a common question asked by many developers who are just starting to look into Akka.NET.

Differences between Akka.NET, Kafka, RabbitMQ, and Other Messaging Systems

Message brokers, enterprise message buses, message queues, event hubs, and so on - for the sake of simplicity, which I’m certain will enrage developers in some corners of the Internet, I’m going to lump these technologies together into a single category: these are message distribution systems.

The manner in which they distribute messages varies and for our purposes those differences are totally immaterial (sorry, vendors.) The point is: producers write messages into these systems and the goal is, with varying degrees of reliability, concurrency, and asynchrony, to distribute these messages for processing to one or more downstream consumers.

Message distribution systems are transports - they aren’t involved in the act of creating or consuming messages. Only in delivering and routing these messages from their sources and to their destinations.

Where Akka.NET differs: Akka.NET actors are fundamentally message processing and message producing technologies.

Akka.NET actors are responsible for managing business state or executing commands, both of which occur when an actor receives a message.

Where the confusion occurs, however,...

Why You Should be Paying Attention to OpenTracing

How the Vendor-Neutral Tracing Standard Will Affect Frameworks and Application Development.

In a previous post we introduced distributed tracing and how it solves some of the worst DevOps problems that arise with the use of microservices. In this post we’re going to introduce the OpenTracing standard and talk how what this project does and why you should pay attention to it if you’re a product owner, software architect, or developer.

Middleware, Frameworks, and Instrumentation

Petabridge is committed to delivering world-class tools for developers to build high-performance, large scale software applications in .NET; thus, developing middleware and application programming frameworks, Akka.NET most notably, is a core part of our mission.

All web and Internet-connected applications depend on frameworks and middleware of some kind, because the overwhelming majority of companies who build software don’t want to be in the business of creating and maintaining infrastructure. This is why we have nice tools like ASP.NET Core, SignalR, Akka.NET, gRPC, DotNetty, NServiceBus, RabbitMQ, and so on.

These frameworks provide us with convenient abstractions that:

  • Significantly lower the cost of developing software in-house;
  • Improve the productivity of individual software developers; and
  • Allow for the formation and sharing of industry best practices and standardized designs.

In short, middleware makes the modern software economy possible.

How Asynchrony, Microservices, and Distributed Systems Changed the Economics

As the Internet has grown, there has been a tremendous amount of pressure on server-side software developers to build systems that are available 24/7/365, capable of working across a larger range of devices (desktop, mobile, embedded, etc…,) able to capture and utilize ever-increasing amounts of data, and capable of responding faster and faster to end-user requests. This pressure is what’s driven the adoption of technologies like the actor model, pushed frameworks like .NET to prioritize the development of asynchronous programming constructs such as the TPL, and driven...

Phobos logo

We released Phobos, our enterprise DevOps suite for Akka.NET only just last month and we’ve already had major Akka.NET users put it through its paces in development and production environments. Based on our users’ feedback and suggestions, we’ve developed Phobos v0.6.0 and made it available for immediate release to all of our evaluation and production users.

If you missed our Phobos announcement here’s the gist of what Phobos does for Akka.NET users:

Phobos instrurments, monitors, and traces all activity from actors inside large Akka.NET applications and exports it to common, off-the-shelf monitoring tools used by .NET enterprises; it works over Akka.Remote and Akka.Cluster; and it can do all of this without any explicit instrumentation code at all. Phobos can be entirely driven through configuration and works automatically behind the scenes.

If you’d like more background information about Phobos, we highly recommend that you visit the Phobos homepage.

Here’s what’s in the newest version of Phobos:

Expanded Tracing and Monitoring Integrations

The biggest addition to Phobos in v0.6.0 is the expanded set of Phobos drivers for working with various monitoring and tracing systems. Here’s what is included in Phobos v0.6.0:

  • Tracing: Jaeger - we’ve added first party support for the popular open source Jaeger tracing engine, which is now available via the Phobos.Tracing.Jaeger NuGet package.
  • Tracing: Microsoft Application Insights - we’ve added OpenTracing-compatible tracing support for Microsoft Azure’s managed Application Insights service via the Phobos.Tracing.ApplicationInsights package.
  • Monitoring: Microsoft Application Insights - in addition to adding tracing support for Application Insights, we’ve also added corresponding monitoring support via the Phobos.Monitoring.ApplicationInsights NuGet package.

If you’re already using the Microsoft Azure platform, we think you’ll find Phobos’ Application Insights integration...

What Happens When Akka.NET Actors Restart

What Gets Lost, What Doesn't, and Why Restarts Help Build Better .NET Applications

One of the most powerful features of Akka.NET actors is their built-in fault tolerance. Actors are an extremely effective tool at isolating failures and preventing them from having side effects on other parts of your application.

However, one of the most frequently asked questions we hear when training .NET developers on the fundamentals of Akka.NET is: “what happens to my data when an actor restarts? Do I lose messages? State?”

We address those questions in this tutorial.

This video covers:

If you like this video please share it and subscribe to Petabridge’s YouTube channel!

How Distributed Tracing Solves One of the Worst Microservices Problems

Distributed Tracing Solves Some Big Pain Points with Microservices

Historically most web applications were developed as monolithic architectures. The entire application shipped as a single process implemented on a single runtime. Ultimately this architectural choice makes scaling software development extremely painstaking and tedious, because 100% of code changes submitted by members of any development team target a single code base. Under monolithic designs deployments are executed as “all or nothing” affairs - either you deploy all parts of your application at once or you deploy none of them.

It’s for painful reasons like this that the industry is moving away from monoliths and towards distributed architectures such as microservices. What microservices provide isn’t product scalability; they provide people scalability - the ability to easily partition your software development organization along the same lines as you partition your application with service boundaries.

An organization partitioned along its microservice boundaries

DevOps Implications of Microservices

Microservices provide enormous agility and flexibility to software development organizations. By partitioning our large applications into interdependent services which communicate via explicit network communication contracts, each team encapsulates their implementation from the others. This makes it possible, in theory, for each team to choose the best tools for the job - if Service 1’s requirements are best satisfied using Node.JS and Redis but Service 2’s are better handled with .NET and SQL Server, both of these teams can make those choices and develop / deploy their services independently of each other.

In practice, microservices are really a trade off for one set of organizational and technical problems for others. While the benefits of microservices amount to greater independence; clearer organizational boundaries and division of labor; and greater agility those benefits come with some distinct costs:

  1. Loss of coherence - now that the work to fulfill a single end-user request is now broken...

Introducing Phobos: Enterprise DevOps Suite for Akka.NET

Zero-code Actor Monitoring and Cross-Cluster Request Tracing

Phobos logo

Today it’s my pleasure to announce the production-ready release of Phobos, an enterprise DevOps suite for Akka.NET developers.

The initial release of Phobos is primarily aimed at solving the following problems:

  1. Instrumenting and monitoring activity from actors inside large Akka.NET applications and exporting it to common, off-the-shelf monitoring tools used by .NET enterprises;
  2. Implementing OpenTracing protocols behind the scenes and over the network stack of Akka.NET, so we can provide end-to-end distributed tracing for Akka.NET actors; and
  3. Doing all of the above automatically without any instrumentation code or setup of any kind. Phobos can be installed into any existing Akka.NET application and capture all of this data with as little as 5 lines of HOCON configuration.

I’d encourage you to watch the “Introduction to Phobos: Enterprise DevOps for Akka.NET” video we put together which explains this in more detail. It’s only about 5 minutes long.

Integration with Third Party Monitoring and Tracing Services

Phobos is designed to act primarily as instrumentation for your Akka.NET applications; it ships all of the metrics and trace data it records to off-the-shelf monitoring and tracing products. This includes a variety of open source and proprietary tools chosen by our customers and users. We know how hard it can be to introduce new tools inside large enterprise environments, so our goal is to be able to support whatever you and your organization currently use.

And if you’ve never used something like Zipkin or StatsD, we’ve collected some docker-compose scripts you can use to test drive those technologies (with or without Phobos) here.

Trying and Using Phobos

Phobos has been in the works for about a year and we’ve...

Akka.Cluster is one of the most popular and useful parts of the Akka.NET ecosystem as a whole, but it’s also one of the most concept-heavy areas. We have a lot of literature on both the official Akka.NET documentation and elsewhere on our blog about concepts such as distributing state in Akka.Cluster, sharding data across cluster nodes using Akka.Cluster.Sharding, publishing messages across a cluster, and so on; however, that barely scratches the surface on the possibilities and uses of Akka.Cluster.

So my goal with this post is to provide a bit of an FAQ on some of the most important and central concepts needed to build and operate effective Akka.NET clusters.

Node Reachability vs. Membership

In Akka.Cluster there are two important, similar-looking concepts that every end-user should be able to distinguish:

  1. Node reachability - is this node available right now? Can I connect to it?
  2. Node membership - is this node a current member of the cluster? Is this node leaving? Joining? Removed?

When many users start working with Akka.Cluster, they operate from the assumption that these two concepts are the same. “If I kill a process that is part of an Akka.NET cluster, that process will no longer be part of the cluster.”

This assumption is incorrect and there’s an important distributed computing concept at work behind this distinction: partition tolerance.

In terms of the CAP theorem, Akka.Cluster provides an AP experience out of the box; Akka.Cluster developers typically trade away some of the cluster’s default availability and partition tolerance (A & P) in exchange for consistency in areas where their domains demand it.

Akka.Cluster’s partition tolerance abilities come from this “reachability” feature; in order to tolerate partitions you have to know where they are and what resources are affected by them...

Petabridge.Cmd and Akka.Monitoring Updated to Support Akka.NET v1.3.1

Petabridge.Cmd Updates to Support .NET Core and .NET Standard

Since the recent Akka.NET v1.3.0 release we’ve been working furiously to upgrade all of our plugins and other tools that are heavily used by the Akka.NET community in order to ensure that you can get everything you need to go into production with confidence.

Today we’re putting out a quick update to make everyone aware of the following updates:

  1. Petabridge.Cmd v0.3.0 and v0.3.1 have been released and now fully supports Akka.NET v1.3.1 as well as .NET Core. As per Petabridge.Cmd’s v0.3.0 release notes, we’ve broken wire compatibility between all previous versions of pbm and Petabridge.Cmd.Host prior to v0.3.0. We did this in order to guarantee that a single pbm client could communicate seamlessly with both .NET Core and .NET 4.* applications. Please make sure you upgrade your pbm clients along with your hosts when you upgrade.
  2. Akka.Monitoring v0.7.0, our open source monitoring extension for Akka.NET applications, is now available and supports Akka.NET v1.3.1. At the moment, however, it does not support .NET Standard - largely due to its dependence on PerformanceCounters and other drivers that do not yet support .NET Standard.

Over the next few months we’re going to be reworking the monitoring story around Akka.NET and Akka.Cluster in order to provide detailed, easy-to-use actor and cluster monitoring instrumentation that works on both .NET Core and .NET 4.5. This is our current top priority and we’ll be providing users on our mailing list with updates once we have something ready to show.

Please let us know in the comments if you have any questions or concerns!

Akka.NET v1.3 is Now Available on .NET 4.5 and .NET Standard 1.6

Akka.NET Adds .NET Standard / .NET Core Support; Releases Akka.Persistence from Beta; and More

In addition to Akka.NET joining the .NET Foundation and announcing our participation in the .NET Foundation Summer Hackfest, we had two more major Akka.NET developments to announce on Friday:

  • Akka.NET v1.3, which adds .NET Standard support to Akka.NET, is released and now available on NuGet and
  • The official Akka.NET website has been totally redone in DocFx in order to make it more readable, user-friendly, and well-organized.

New Akka.NET logo

Akka.NET v1.3 has been a work in progress since .NET Core was first released last year and is the culmination of hundreds of commits by dozens of contributors. You can read the official Akka.NET v1.3 release notes here, but what follows are my notes on some of the changes end-users can expect when adopting v1.3 and upgrading from earlier versions of Akka.NET.

.NET Standard 1.6 Support for Akka.NET

The most-requested feature for Akka.NET over the past year has been .NET Standard / .NET Core support.

In case you’re not sure what the difference is between .NET Standard and .NET Core: .NET Standard is a compilation target shared by both the full .NET framework (4.6.*) and .NET Core. Any library that targets .NET Standard 1.6, for instance, can be used inside .NET 4.6.1 AND .NET Core 1.1 applications. You can read more about .NET Standard and the various versions of it here at Microsoft’s official documentation.

Akka.NET’s NuGet packages now target both .NET 4.5 (which we always have, historically) and .NET Standard 1.6. NuGet will automatically install the .NET Standard 1.6 version of the NuGet package if you’re targeting .NET 4.6.1 or .NET Core.

We’ve already been using the Akka.NET v1.3 nightly builds in a new production-facing project and the results have been excellent thus far and we look forward...

Akka.NET has Joined the .NET Foundation

And Also: Introducing the AkkaNetContrib Organization and the Akka.NET Summer Hackfest

It’s our pleasure to announce that as of today, Akka.NET is officially part of the .NET Foundation! Akka.NET is joining an elite group of projects such as Roslyn, NuGet, .NET Core, ASP.NET Core, and others.

New Akka.NET logo

What Does This Mean for Akka.NET?

Open source foundations have been a mainstay of OSS projects across many different platforms for years; ultimately what they provide to OSS projects are legal support, marketing help, and general project guidance. This includes things like protecting the project’s copyrights, ensuring that its intellectual property is in good standing and is properly assigned, etc.

In general the goal of the .NET Foundation is to help ensure that all of its projects follow OSS best practices in order to increase adoption of OSS across the entire .NET ecosystem.

Akka.NET already has major users such as Bank of America, SNL Financial, Schneider Electric, Boeing, and others. And the rate at which Akka.NET is being installed and deployed into production is increasing all the time. We’re joining the .NET Foundation in order to help broaden the tent of Akka.NET users and send a signal to potential users everywhere that Akka.NET is in it for the long haul and that our project is enterprise-friendly.

The core development team behind Akka.NET will continue to do what they’ve been doing; our roadmap and our priorities are driven by achieving parity with our partners at Lightbend and the Scala implementation of Akka and by the demands of our users. Joining the .NET Foundation will enable us to focus on our code, samples, and documentation while it helps us keep the project’s IP and community standards in good hands.

Announcing the Akka.NET Contrib Organization and the 2017 .NET Foundation Summer Hackfest

One other big change we wish to...