Performance Testing Should be Mandatory

Plus, How to Actually do it Right

Back in December I released the first publicly available version of NBench - Petabridge’s automated .NET performance testing and benchmarking framework.

NBench Logo

NBench has proven itself to be an invaluable part of our QA process for Akka.NET and scores of other projects, for one critical reason: performance is a mission critical feature for an increasingly large number of applications. And if you can’t measure performance, then you’re shipping a totally untested feature to your end-users.

The Impact of Poor Performance

A simple anecdote to illustrate the real-world impact of shipping non-performant software.

One of my favorite musicians, whom I have never seen live before, is coming to town and I wanted to purchase a ticket. I was out of the country the day tickets went on sale, so if I wanted to attend I’d need to buy a ticket from a secondary market.

I decided to give a new company I’d never purchased tickets through before, SeatGeek, a try. I quickly found two tickets for about $100 each, went through the checkout process, put in my payment information, and submitted payment. A few seconds later I get an error message back letting me know that the tickets were no longer available. So I repeat this process a few more times with progressively more expensive tickets with no success.

Eventually I just gave up and SeatGeek lost about $500 worth of revenue, because I lost any confidence that their reported ticket inventory was available. The crucial error was that the underlying software responsible for reporting inventory availability under-performed - it wasn’t able to keep up with the demand of actual customers, and as a result they nearly lost my business. I tried again on a whim immediately before writing this post and...

This is a follow-up to our last post, “How to Guarantee Delivery of Messages in Akka.NET.” In that post we introduced the AtLeastOnceDeliveryActor and talked about strategies for retrying deliveries of messages in order to make sure they’re successfully procssed.

In this post we’re going to talk about “At Least Once” delivery’s big brother, “Exactly Once” delivery - and why you should try to avoid it if at all possible.

A question we get asked about frequently with Akka.NET is “how can I guarantee that my messages will eventually be received and processed by another actor?” Most of the time it’s developers who work on finance applications who have this question, because each missed message means that someone’s accounting is off - but developers in lots of other domains have this issue too.