Introduction to Akka.Streams

Building High-Level Streaming Worklows in Akka.NET

The goal of this blog post is to give you a glimpse of the idea and basics behind Akka.Streams. What they are and why you may find them useful addition in your day to day job.

In short: Akka.Streams is a library build on top of Akka.NET, which allows you to consume and process potentially infinite streams of data in type-safe and resource-safe way.

To make it easier to visualize, let’s take an example: we have a queue (i.e. RabbitMQ), that sends us a documents, which we have to parse, turn into structured data and save in the database.

How to Use Github Professionally

Best Practices for Working with Github in Team Settings

I originally started writing this post as internal documentation for our own team here at Petabridge, but I thought this would be useful for our readers and users as well.

Github has evolved over the years into a vast, rich ecosystem filled with lots of first and third party features that make developers more productive and effective.

Yet the vast majority of developers haven’t had much experience working effectively with Github in day-to-day work. Many developers don’t have a Github account; some have created some simple projects or filed some bug reports on popular projects; and few have forked a repository and made a pull request.

In this post you’re going to learn the best practices for working with teams of developers on Github who are working towards producing production-ready software. Everything in this post is equally applicable for developers working behind the firewall on proprietary software via Github Enterprise as it is for developers who want to submit a patch to popular open source projects like Akka.NET.

Two Unnecessary Costs of Software Development

Putting my “Chief Technology Officer” hat on for a second, there are lots of cost levers behind the total expense of software development and most of them are necessities. Yes, we should always allow plenty of time and money for testing and user feedback. Yes, we should try to pay down technical debt. We’re not talking about any of that.

What I’m talking about are unnecessary costs, waste costs, that can be avoided via using Github effectively as a communication platform among a development team. Those costs are:

  1. False starts - designing the wrong thing from the beginning;
  2. Blind alleys - designing the right thing using the wrong strategy.

In both of these cases the developers’ time and company money is wasted....

Best Practices for Designing Akka.NET Domain Events and Commands

How to Make Akka.NET Programming Easier by Designing Events Well

In this blog post we’re going to cover some best practices you can use when designing domain events and objects intended to work with Akka.NET. If you follow these best practices you’ll run into fewer errors, clearer log messages, and a better extensibility experience.

Use Marker / Identity Interfaces Generously

This first tip is designed to make it easier to extend your messaging systems without having to manually update the Receive<T> statements on a large number of actors.

Suppose I’m working the code from our new Akka.Cluster Workshop - in this application we have a large number of domain events for the purpose of trading stocks:

  1. Bid - offer to buy N units of stock at a specific price point;
  2. Ask - offer to sell N units of stock at a specific price point;
  3. Match - the stock trading system has matched an Ask order with a Bid; and
  4. Fill - some amount of a Bid or Ask order has been filled by a Match.

All of these events have several common identifiers and properties that can be really useful for routing, sharding, or distributing these messages:

  1. The stock ticker symbol (MSFT, TEAM, AMD, etc…);
  2. The id of the order; and
  3. They all represent live trading events happening as a result of trader activity - this is distinct from an event emitted by the exchange indicating what the newest “market price” for a specific ticker symbol is.

Well, in order to make my system more extensible and easier to debug I’m going to introduce some common marker interfaces - for instance, an IWithStockId interface:

/// <summary> 
    
  

New Akka.Remote and Akka.Cluster.Sharding Command Palettes for Petabridge.Cmd

Tools for Akka.Cluster.Sharding Management and Akka.Remote Management

We introduced Petabridge.Cmd in 2017 to make it easy for Akka.NET users to deploy, monitor, and manage their Akka.NET applications. We have now expanded the capabilities of Petabridge.Cmd by introducing two new command palettes to further simplify the way you monitor your Akka.NET applications. The new command palettes will allow you to monitor your system connections with Petabridge.Cmd.Remote and allow you to obtain sharding information using Petabridge.Cmd.Cluster.Sharding.

If you have not heard of Petabridge.Cmd yet, please take a look at our introduction YouTube video for Petabridge.Cmd

Video Highlights

  • Description of Petabridge.Cmd.
  • How Petabridge.Cmd works.
  • How Petabridge.Cmd uses command palettes.
  • Demo of setting up Petabridge.Cmd and the use of commands.

Petabridge.Cmd.Remote Commands

Let’s cover the 4 new commands that we have introduced with Petabridge.Cmd.Remote. These can be used with applications using Akka.Remote or Akka.Cluster:

  1. remote stats - allows you to see the number of Association and Disassociation events seen by the node running your Petabridge.Cmd.Host;
  2. remote connections - provides you with the number of active remote connections to the local host;
  3. remote tail - writes all of the connections and disconnections events out to the console until stopped via Control + C and;
  4. remote history - shows the connection events witnessed by the node. Can show up to 1,000 events.

We encourage you to give these commands a try in your application. Below you can see the output of some of the commands in our remote command palette. We ran the remote connections on a cluster and you can see the connection of our host node to all of the remote nodes. We also ran our remote history command with...

Akka.NET vs. Kafka, RabbitMQ, and Other Messaging Systems

What's the difference between Akka.NET, Kafka, RabbitMQ, and other message-driven technologies? Can they work together?

A while back I created a thread on Twitter to attempt to explain the difference between Akka.NET and some other popular message-distribution and queuing technologies, such as Apache Kafka and RabbitMQ.

I’m going to cover that in some more detail in this post because it’s a common question asked by many developers who are just starting to look into Akka.NET.

Differences between Akka.NET, Kafka, RabbitMQ, and Other Messaging Systems

Message brokers, enterprise message buses, message queues, event hubs, and so on - for the sake of simplicity, which I’m certain will enrage developers in some corners of the Internet, I’m going to lump these technologies together into a single category: these are message distribution systems.

The manner in which they distribute messages varies and for our purposes those differences are totally immaterial (sorry, vendors.) The point is: producers write messages into these systems and the goal is, with varying degrees of reliability, concurrency, and asynchrony, to distribute these messages for processing to one or more downstream consumers.

Message distribution systems are transports - they aren’t involved in the act of creating or consuming messages. Only in delivering and routing these messages from their sources and to their destinations.

Where Akka.NET differs: Akka.NET actors are fundamentally message processing and message producing technologies.

Akka.NET actors are responsible for managing business state or executing commands, both of which occur when an actor receives a message.

Where the confusion occurs, however,...

Why You Should be Paying Attention to OpenTracing

How the Vendor-Neutral Tracing Standard Will Affect Frameworks and Application Development.

In a previous post we introduced distributed tracing and how it solves some of the worst DevOps problems that arise with the use of microservices. In this post we’re going to introduce the OpenTracing standard and talk how what this project does and why you should pay attention to it if you’re a product owner, software architect, or developer.

Middleware, Frameworks, and Instrumentation

Petabridge is committed to delivering world-class tools for developers to build high-performance, large scale software applications in .NET; thus, developing middleware and application programming frameworks, Akka.NET most notably, is a core part of our mission.

All web and Internet-connected applications depend on frameworks and middleware of some kind, because the overwhelming majority of companies who build software don’t want to be in the business of creating and maintaining infrastructure. This is why we have nice tools like ASP.NET Core, SignalR, Akka.NET, gRPC, DotNetty, NServiceBus, RabbitMQ, and so on.

These frameworks provide us with convenient abstractions that:

  • Significantly lower the cost of developing software in-house;
  • Improve the productivity of individual software developers; and
  • Allow for the formation and sharing of industry best practices and standardized designs.

In short, middleware makes the modern software economy possible.

How Asynchrony, Microservices, and Distributed Systems Changed the Economics

As the Internet has grown, there has been a tremendous amount of pressure on server-side software developers to build systems that are available 24/7/365, capable of working across a larger range of devices (desktop, mobile, embedded, etc…,) able to capture and utilize ever-increasing amounts of data, and capable of responding faster and faster to end-user requests. This pressure is what’s driven the adoption of technologies like the actor model, pushed frameworks like .NET to prioritize the development of asynchronous programming constructs such as the TPL, and driven...

Phobos logo

We released Phobos, our enterprise DevOps suite for Akka.NET only just last month and we’ve already had major Akka.NET users put it through its paces in development and production environments. Based on our users’ feedback and suggestions, we’ve developed Phobos v0.6.0 and made it available for immediate release to all of our evaluation and production users.

If you missed our Phobos announcement here’s the gist of what Phobos does for Akka.NET users:

Phobos instrurments, monitors, and traces all activity from actors inside large Akka.NET applications and exports it to common, off-the-shelf monitoring tools used by .NET enterprises; it works over Akka.Remote and Akka.Cluster; and it can do all of this without any explicit instrumentation code at all. Phobos can be entirely driven through configuration and works automatically behind the scenes.

If you’d like more background information about Phobos, we highly recommend that you visit the Phobos homepage.

Here’s what’s in the newest version of Phobos:

Expanded Tracing and Monitoring Integrations

The biggest addition to Phobos in v0.6.0 is the expanded set of Phobos drivers for working with various monitoring and tracing systems. Here’s what is included in Phobos v0.6.0:

  • Tracing: Jaeger - we’ve added first party support for the popular open source Jaeger tracing engine, which is now available via the Phobos.Tracing.Jaeger NuGet package.
  • Tracing: Microsoft Application Insights - we’ve added OpenTracing-compatible tracing support for Microsoft Azure’s managed Application Insights service via the Phobos.Tracing.ApplicationInsights package.
  • Monitoring: Microsoft Application Insights - in addition to adding tracing support for Application Insights, we’ve also added corresponding monitoring support via the Phobos.Monitoring.ApplicationInsights NuGet package.

If you’re already using the Microsoft Azure platform, we think you’ll find Phobos’ Application Insights integration...

What Happens When Akka.NET Actors Restart

What Gets Lost, What Doesn't, and Why Restarts Help Build Better .NET Applications

One of the most powerful features of Akka.NET actors is their built-in fault tolerance. Actors are an extremely effective tool at isolating failures and preventing them from having side effects on other parts of your application.

However, one of the most frequently asked questions we hear when training .NET developers on the fundamentals of Akka.NET is: “what happens to my data when an actor restarts? Do I lose messages? State?”

We address those questions in this tutorial.

This video covers:

If you like this video please share it and subscribe to Petabridge’s YouTube channel!

How Distributed Tracing Solves One of the Worst Microservices Problems

Distributed Tracing Solves Some Big Pain Points with Microservices

Historically most web applications were developed as monolithic architectures. The entire application shipped as a single process implemented on a single runtime. Ultimately this architectural choice makes scaling software development extremely painstaking and tedious, because 100% of code changes submitted by members of any development team target a single code base. Under monolithic designs deployments are executed as “all or nothing” affairs - either you deploy all parts of your application at once or you deploy none of them.

It’s for painful reasons like this that the industry is moving away from monoliths and towards distributed architectures such as microservices. What microservices provide isn’t product scalability; they provide people scalability - the ability to easily partition your software development organization along the same lines as you partition your application with service boundaries.

An organization partitioned along its microservice boundaries

DevOps Implications of Microservices

Microservices provide enormous agility and flexibility to software development organizations. By partitioning our large applications into interdependent services which communicate via explicit network communication contracts, each team encapsulates their implementation from the others. This makes it possible, in theory, for each team to choose the best tools for the job - if Service 1’s requirements are best satisfied using Node.JS and Redis but Service 2’s are better handled with .NET and SQL Server, both of these teams can make those choices and develop / deploy their services independently of each other.

In practice, microservices are really a trade off for one set of organizational and technical problems for others. While the benefits of microservices amount to greater independence; clearer organizational boundaries and division of labor; and greater agility those benefits come with some distinct costs:

  1. Loss of coherence - now that the work to fulfill a single end-user request is now broken...

Introducing Phobos: Enterprise DevOps Suite for Akka.NET

Zero-code Actor Monitoring and Cross-Cluster Request Tracing

Phobos logo

Today it’s my pleasure to announce the production-ready release of Phobos, an enterprise DevOps suite for Akka.NET developers.

The initial release of Phobos is primarily aimed at solving the following problems:

  1. Instrumenting and monitoring activity from actors inside large Akka.NET applications and exporting it to common, off-the-shelf monitoring tools used by .NET enterprises;
  2. Implementing OpenTracing protocols behind the scenes and over the network stack of Akka.NET, so we can provide end-to-end distributed tracing for Akka.NET actors; and
  3. Doing all of the above automatically without any instrumentation code or setup of any kind. Phobos can be installed into any existing Akka.NET application and capture all of this data with as little as 5 lines of HOCON configuration.

I’d encourage you to watch the “Introduction to Phobos: Enterprise DevOps for Akka.NET” video we put together which explains this in more detail. It’s only about 5 minutes long.

Integration with Third Party Monitoring and Tracing Services

Phobos is designed to act primarily as instrumentation for your Akka.NET applications; it ships all of the metrics and trace data it records to off-the-shelf monitoring and tracing products. This includes a variety of open source and proprietary tools chosen by our customers and users. We know how hard it can be to introduce new tools inside large enterprise environments, so our goal is to be able to support whatever you and your organization currently use.

And if you’ve never used something like Zipkin or StatsD, we’ve collected some docker-compose scripts you can use to test drive those technologies (with or without Phobos) here.

Trying and Using Phobos

Phobos has been in the works for about a year and we’ve...