The Current State of OpenTelemetry in .NET

OpenTelemetry vs. OpenTracing and the Future of Observability in .NET

In the past we’ve discussed why distributed tracing is becoming commonplace and the OpenTracing standard for instrumenting libraries and applications. In this post I want to touch on the emerging OpenTelemetry standard, which will become a common component used to instrument ASP.NET Core applications in the not too distant future.

OpenTelemetry logo

What is OpenTelemetry?

OpenTelemetry is the convergence of two competing tracing standards:

  • OpenTracing - developed by a community of APM vendors and library authors and
  • OpenCensus - developed by Google.

The goal is to provide a unified set of APIs library authors can include inside their applications in order to:

  • Propagate distributed tracing context, including the new W3C HTTP tracing standard;
  • Aggregate metrics (counters, meters, etc); and
  • Export metrics and trace data to a variety of different Application Performance Monitoring (APM) backends, which can be configured entirely by the application developer.

OpenTelemetry vs. OpenTracing

OpenTelemetry vs. OpenTracing

So what are the material differences between OpenTelemetry and OpenTracing? Why do we need another new standard?

The major technical differences are:

  1. OpenTelemetry’s core library is the Tracer implementation - the traces are created and correlated using OpenTelemetry calls and then only during the export process do the traces hit any vendor-specific code. This makes the performance of OpenTelemetry very consistent regardless of what vendors end-users choose. In contrast, with OpenTracing all of the real calls are done by a vendor-specific implementation of the OpenTracing APIs - so as a library author you could have a great set of benchmarks using a Zipkin OpenTracing library but not-so-great ones using a Jaeger OpenTracing library. I prefer OpenTelemetry’s approach here.
  2. OpenTelemetry supports metrics instrumentation in addition to tracing - a library author...

Akka.NET v1.4 is Available on .NET Standard 2.0

Akka.Remote Performance Improvements, Akka.Cluster.Sharding out of Beta, Akka.Cluster.Metrics, and More

As of this week, Akka.NET v1.4 is now fully available for users to use. We’ve published a detailed article on the Akka.NET website that describes what’s new in Akka.NET v1.4, but we wanted to capture some of the highlights here.

Akka.Cluster.Sharding and Akka.DistributedData are out of Beta

Akka.Cluster.Sharding has been used by Akka.NET users in production for years, mostly relying on the underlying Akka.Persistence storage engine to save all of the sharding state used to distributed entity actors evenly across the cluster.

Akka.Cluster.Sharding actor distribution

However, the reason why the module remained in beta for the past couple of years is because Akka.Cluster.Sharding’s alternate storage mode, Akka.DistributedData, which uses eventually consistent in-memory replication to manage cluster sharding state throughout an Akka.NET cluster, lacked a stable wire format and hadn’t been certified for production yet.

Introduction to Akka.Streams

Building High-Level Streaming Worklows in Akka.NET

The goal of this blog post is to give you a glimpse of the idea and basics behind Akka.Streams. What they are and why you may find them useful addition in your day to day job.

In short: Akka.Streams is a library build on top of Akka.NET, which allows you to consume and process potentially infinite streams of data in type-safe and resource-safe way.

To make it easier to visualize, let’s take an example: we have a queue (i.e. RabbitMQ), that sends us a documents, which we have to parse, turn into structured data and save in the database.

How to Use Github Professionally

Best Practices for Working with Github in Team Settings

I originally started writing this post as internal documentation for our own team here at Petabridge, but I thought this would be useful for our readers and users as well.

Github has evolved over the years into a vast, rich ecosystem filled with lots of first and third party features that make developers more productive and effective.

Yet the vast majority of developers haven’t had much experience working effectively with Github in day-to-day work. Many developers don’t have a Github account; some have created some simple projects or filed some bug reports on popular projects; and few have forked a repository and made a pull request.

In this post you’re going to learn the best practices for working with teams of developers on Github who are working towards producing production-ready software. Everything in this post is equally applicable for developers working behind the firewall on proprietary software via Github Enterprise as it is for developers who want to submit a patch to popular open source projects like Akka.NET.

Two Unnecessary Costs of Software Development

Putting my “Chief Technology Officer” hat on for a second, there are lots of cost levers behind the total expense of software development and most of them are necessities. Yes, we should always allow plenty of time and money for testing and user feedback. Yes, we should try to pay down technical debt. We’re not talking about any of that.

What I’m talking about are unnecessary costs, waste costs, that can be avoided via using Github effectively as a communication platform among a development team. Those costs are:

  1. False starts - designing the wrong thing from the beginning;
  2. Blind alleys - designing the right thing using the wrong strategy.

In both of these cases the developers’ time and company’s money is wasted....

Best Practices for Designing Akka.NET Domain Events and Commands

How to Make Akka.NET Programming Easier by Designing Events Well

In this blog post we’re going to cover some best practices you can use when designing domain events and objects intended to work with Akka.NET. If you follow these best practices you’ll run into fewer errors, clearer log messages, and a better extensibility experience.

Use Marker / Identity Interfaces Generously

This first tip is designed to make it easier to extend your messaging systems without having to manually update the Receive<T> statements on a large number of actors.

Suppose I’m working the code from our new Akka.Cluster Workshop - in this application we have a large number of domain events for the purpose of trading stocks:

  1. Bid - offer to buy N units of stock at a specific price point;
  2. Ask - offer to sell N units of stock at a specific price point;
  3. Match - the stock trading system has matched an Ask order with a Bid; and
  4. Fill - some amount of a Bid or Ask order has been filled by a Match.

All of these events have several common identifiers and properties that can be really useful for routing, sharding, or distributing these messages:

  1. The stock ticker symbol (MSFT, TEAM, AMD, etc…);
  2. The id of the order; and
  3. They all represent live trading events happening as a result of trader activity - this is distinct from an event emitted by the exchange indicating what the newest “market price” for a specific ticker symbol is.

Well, in order to make my system more extensible and easier to debug I’m going to introduce some common marker interfaces - for instance, an IWithStockId interface:

/// <summary> 
    
  

New Akka.Remote and Akka.Cluster.Sharding Command Palettes for Petabridge.Cmd

Tools for Akka.Cluster.Sharding Management and Akka.Remote Management

We introduced Petabridge.Cmd in 2017 to make it easy for Akka.NET users to deploy, monitor, and manage their Akka.NET applications. We have now expanded the capabilities of Petabridge.Cmd by introducing two new command palettes to further simplify the way you monitor your Akka.NET applications. The new command palettes will allow you to monitor your system connections with Petabridge.Cmd.Remote and allow you to obtain sharding information using Petabridge.Cmd.Cluster.Sharding.

If you have not heard of Petabridge.Cmd yet, please take a look at our introduction YouTube video for Petabridge.Cmd

Video Highlights

  • Description of Petabridge.Cmd.
  • How Petabridge.Cmd works.
  • How Petabridge.Cmd uses command palettes.
  • Demo of setting up Petabridge.Cmd and the use of commands.

Petabridge.Cmd.Remote Commands

Let’s cover the 4 new commands that we have introduced with Petabridge.Cmd.Remote. These can be used with applications using Akka.Remote or Akka.Cluster:

  1. remote stats - allows you to see the number of Association and Disassociation events seen by the node running your Petabridge.Cmd.Host;
  2. remote connections - provides you with the number of active remote connections to the local host;
  3. remote tail - writes all of the connections and disconnections events out to the console until stopped via Control + C and;
  4. remote history - shows the connection events witnessed by the node. Can show up to 1,000 events.

We encourage you to give these commands a try in your application. Below you can see the output of some of the commands in our remote command palette. We ran the remote connections on a cluster and you can see the connection of our host node to all of the remote nodes. We also ran our remote history command with...

Akka.NET vs. Kafka, RabbitMQ, and Other Messaging Systems

What's the difference between Akka.NET, Kafka, RabbitMQ, and other message-driven technologies? Can they work together?

A while back I created a thread on Twitter to attempt to explain the difference between Akka.NET and some other popular message-distribution and queuing technologies, such as Apache Kafka and RabbitMQ.

I’m going to cover that in some more detail in this post because it’s a common question asked by many developers who are just starting to look into Akka.NET.

Differences between Akka.NET, Kafka, RabbitMQ, and Other Messaging Systems

Message brokers, enterprise message buses, message queues, event hubs, and so on - for the sake of simplicity, which I’m certain will enrage developers in some corners of the Internet, I’m going to lump these technologies together into a single category: these are message distribution systems.

The manner in which they distribute messages varies and for our purposes those differences are totally immaterial (sorry, vendors.) The point is: producers write messages into these systems and the goal is, with varying degrees of reliability, concurrency, and asynchrony, to distribute these messages for processing to one or more downstream consumers.

Message distribution systems are transports - they aren’t involved in the act of creating or consuming messages. Only in delivering and routing these messages from their sources and to their destinations.

Where Akka.NET differs: Akka.NET actors are fundamentally message processing and message producing technologies.

Akka.NET actors are responsible for managing business state or executing commands, both of which occur when an actor receives a message.

Where the confusion occurs, however,...

Why You Should be Paying Attention to OpenTracing

How the Vendor-Neutral Tracing Standard Will Affect Frameworks and Application Development.

In a previous post we introduced distributed tracing and how it solves some of the worst DevOps problems that arise with the use of microservices. In this post we’re going to introduce the OpenTracing standard and talk how what this project does and why you should pay attention to it if you’re a product owner, software architect, or developer.

Middleware, Frameworks, and Instrumentation

Petabridge is committed to delivering world-class tools for developers to build high-performance, large scale software applications in .NET; thus, developing middleware and application programming frameworks, Akka.NET most notably, is a core part of our mission.

All web and Internet-connected applications depend on frameworks and middleware of some kind, because the overwhelming majority of companies who build software don’t want to be in the business of creating and maintaining infrastructure. This is why we have nice tools like ASP.NET Core, SignalR, Akka.NET, gRPC, DotNetty, NServiceBus, RabbitMQ, and so on.

These frameworks provide us with convenient abstractions that:

  • Significantly lower the cost of developing software in-house;
  • Improve the productivity of individual software developers; and
  • Allow for the formation and sharing of industry best practices and standardized designs.

In short, middleware makes the modern software economy possible.

How Asynchrony, Microservices, and Distributed Systems Changed the Economics

As the Internet has grown, there has been a tremendous amount of pressure on server-side software developers to build systems that are available 24/7/365, capable of working across a larger range of devices (desktop, mobile, embedded, etc…,) able to capture and utilize ever-increasing amounts of data, and capable of responding faster and faster to end-user requests. This pressure is what’s driven the adoption of technologies like the actor model, pushed frameworks like .NET to prioritize the development of asynchronous programming constructs such as the TPL, and driven...

Phobos logo

We released Phobos, our enterprise DevOps suite for Akka.NET only just last month and we’ve already had major Akka.NET users put it through its paces in development and production environments. Based on our users’ feedback and suggestions, we’ve developed Phobos v0.6.0 and made it available for immediate release to all of our evaluation and production users.

If you missed our Phobos announcement here’s the gist of what Phobos does for Akka.NET users:

Phobos instrurments, monitors, and traces all activity from actors inside large Akka.NET applications and exports it to common, off-the-shelf monitoring tools used by .NET enterprises; it works over Akka.Remote and Akka.Cluster; and it can do all of this without any explicit instrumentation code at all. Phobos can be entirely driven through configuration and works automatically behind the scenes.

If you’d like more background information about Phobos, we highly recommend that you visit the Phobos homepage.

Here’s what’s in the newest version of Phobos:

Expanded Tracing and Monitoring Integrations

The biggest addition to Phobos in v0.6.0 is the expanded set of Phobos drivers for working with various monitoring and tracing systems. Here’s what is included in Phobos v0.6.0:

  • Tracing: Jaeger - we’ve added first party support for the popular open source Jaeger tracing engine, which is now available via the Phobos.Tracing.Jaeger NuGet package.
  • Tracing: Microsoft Application Insights - we’ve added OpenTracing-compatible tracing support for Microsoft Azure’s managed Application Insights service via the Phobos.Tracing.ApplicationInsights package.
  • Monitoring: Microsoft Application Insights - in addition to adding tracing support for Application Insights, we’ve also added corresponding monitoring support via the Phobos.Monitoring.ApplicationInsights NuGet package.

If you’re already using the Microsoft Azure platform, we think you’ll find Phobos’ Application Insights integration...

What Happens When Akka.NET Actors Restart

What Gets Lost, What Doesn't, and Why Restarts Help Build Better .NET Applications

One of the most powerful features of Akka.NET actors is their built-in fault tolerance. Actors are an extremely effective tool at isolating failures and preventing them from having side effects on other parts of your application.

However, one of the most frequently asked questions we hear when training .NET developers on the fundamentals of Akka.NET is: “what happens to my data when an actor restarts? Do I lose messages? State?”

We address those questions in this tutorial.

This video covers:

If you like this video please share it and subscribe to Petabridge’s YouTube channel!