Performance Testing Should be Mandatory

Plus, How to Actually do it Right

Back in December I released the first publicly available version of NBench - Petabridge’s automated .NET performance testing and benchmarking framework.

NBench Logo

NBench has proven itself to be an invaluable part of our QA process for Akka.NET and scores of other projects, for one critical reason: performance is a mission critical feature for an increasingly large number of applications. And if you can’t measure performance, then you’re shipping a totally untested feature to your end-users.

The Impact of Poor Performance

A simple anecdote to illustrate the real-world impact of shipping non-performant software.

One of my favorite musicians, whom I have never seen live before, is coming to town and I wanted to purchase a ticket. I was out of the country the day tickets went on sale, so if I wanted to attend I’d need to buy a ticket from a secondary market.

I decided to give a new company I’d never purchased tickets through before, SeatGeek, a try. I quickly found two tickets for about $100 each, went through the checkout process, put in my payment information, and submitted payment. A few seconds later I get an error message back letting me know that the tickets were no longer available. So I repeat this process a few more times with progressively more expensive tickets with no success.

Eventually I just gave up and SeatGeek lost about $500 worth of revenue, because I lost any confidence that their reported ticket inventory was available. The crucial error was that the underlying software responsible for reporting inventory availability under-performed - it wasn’t able to keep up with the demand of actual customers, and as a result they nearly lost my business. I tried again on a whim immediately before writing this post and...

This is a follow-up to our last post, “How to Guarantee Delivery of Messages in Akka.NET.” In that post we introduced the AtLeastOnceDeliveryActor and talked about strategies for retrying deliveries of messages in order to make sure they’re successfully procssed.

In this post we’re going to talk about “At Least Once” delivery’s big brother, “Exactly Once” delivery - and why you should try to avoid it if at all possible.

A question we get asked about frequently with Akka.NET is “how can I guarantee that my messages will eventually be received and processed by another actor?” Most of the time it’s developers who work on finance applications who have this question, because each missed message means that someone’s accounting is off - but developers in lots of other domains have this issue too.

After last month’s successful Akka.NET virtual meetup, we’re going to be coming right back with another on on February 29th at 18:00 UTC. RSVP Here.

Update: thanks to everyone who attended! You can view the meetup below!

Petabridge does .NET open source software. One of the great things that’s happening in .NET-land right now is that scores of developers from traditionally more conservative, pure Microsoft stack companies are starting to become not just users of open source software, but also contributors!

Since lots of .NET developers are new to Github, having been previously raised on a steady diet of Perforce and Team Foundation Server, I put this video together illustrating how to do all of the Github workflows used by most major .NET OSS projects.

2016 is going to be a big year for Akka.NET, and we’re starting things off right by hosting our second Akka.NET Virtual Meetup on Tuesday, January 26th @ 18:00UTC. RSVP Here.

Update: thanks to everyone who attended! You can view the meetup below!

Creating Persistent Actors in Akka.NET with Akka.Persistence

How to Create Akka.NET Actors with Durable State

One of the most frequently asked questions I get is about how to create stateful actors that are also durable by default, meaning that the actor can recover its state from some sort of storage engine in the event that the entire Akka.NET process needs to be restarted.

Enter Akka.Persistence - an entire framework built into Akka.NET that’s designed to allow you to create actors with durable state that can be persisted on any database or storage system you want.

In this post we’re going to explore the concepts behind Akka.Persistence, how it works, and what some of the available storage options are.

Not long ago in Akka.NET-land we had an issue occur where users noticed a dramatic drop in throughput in Akka.Remote’s message processing pipeline - and to make matters worse, this occurred in a production release of AKka.NET!

Yikes, how did that happen?

The answer is that although you can use unit tests and code reviews to detect functional problems with code changes and pull requests, using those same mechanisms to detect performance problems with code is utterly ineffective. Even skilled developers who have detailed knowledge about the internals of the .NET framework and CLR are unable to correctly predict how changes to code will impact its performance.

Hence why I developed NBench - a .NET performance-testing, stress-testing, and benchmarking framework for .NET applications that works and feels a lot like a unit test.

How to Unit Test Akka.NET Actors with Akka.TestKit

An Introduction to the Akka.TestKit

In this post we introduce the Akka.TestKit module - a library that makes it easy to unit test Akka.NET actors using all of the popular .NET unit testing frameworks.

A Brief End-to-End Akka.TestKit Example

Before going deep into the TestKit, here’s an end-to-end example of what a test usually looks like.

[TestFixture] //using NUnit
public class UserIdentityActorSpecs : TestKit{

    [Test]
    public void UserIdentityActor_should_confirm_user_creation_success()
    {
        var identity = Sys.ActorOf(Props.Create(() => new UserIdentityActor()));
        identity.Tell(new UserIdentityActor.CreateUserWithValidUserInfo());
        var result = ExpectMsg<UserIdentityActor.OperationResult>().Successful;
        Assert.True(result);
    }

}

80% of your tests will be this simple: create an actor, send it a message, and expect a response back. Let’s explore how to do it.

The concepts in this post were first introduced in a talk at the 2015 Cassandra Summit.

I’ve been a .NET developer for roughly 10 years now - since the summer after my freshman year in college in 2005 I’ve been developing in Visual Studio and .NET. I’ve founded three startups on .NET, worked for Microsoft, and founded multiple successful OSS projects in .NET - I say all of this in evidence to the depth of my commitment and investment in the .NET ecosystem.

But it’s time we, the .NET community, address a major elephant in the room - .NET is, and often has been, way behind in other ecosystems in terms of overall innovation, openness to new ideas, and flexibility.

The Experience of Being a .NET Developer

.NET has been, historically, an expensive ecosystem to play in.

Because to play in it, you have to do so by Microsoft’s rules.

It's Ballmer's World, Bro

  • You have to buy a license for Visual Studio, or more likely, an MSDN subscription.
  • You have to buy a license for Windows Server.
  • You have to buy a license for Windows Server.
  • You have to develop your web applications in a framework built by Microsoft, like ASP.NET or WCF.
  • You have to host your web applications in IIS.

And the list goes on - the point being that our entire way of developing our own products depends on a single company in Redmond choosing what tools it wants to make available to us at any given time and price.

The Impact of Marrying .NET to Microsoft

I want to share with you a little story from a point in my career with Microsoft during 2010-2011. By 2011 HTML5 had become mostly standardized and its...