Reactive .NET Pipelines with Backpressure

// WHEN_TO_USE_IT

Choose the Right .NET Pipeline

Use Akka.Streams when a multi-stage pipeline needs backpressure, bounded resource use, supervision, or connectors such as Kafka and RabbitMQ. Use IAsyncEnumerable, Channels, or TPL Dataflow for a smaller in-process pipeline when you do not need a full stream graph. Use a queue and background workers when work must survive process restarts or be scheduled independently of the request that created it.

// SOURCE_FLOW_SINK

Composable Pipeline Architecture

Build processing graphs by composing Sources, Flows, and Sinks. Each stage is strongly typed, independently configurable, and backpressure-aware.

Source produces, Flow transforms, Sink consumes — backpressure flows upstream automatically

// BACKPRESSURE

Automatic Flow Control

When downstream can't keep up, the entire pipeline throttles automatically. No dropped messages, no manual flow control code.

Database slows down → Request(n) propagates upstream → Source pauses → resumes when recovered

// IN_CODE

Looks Like LINQ, Runs Like a Pipeline

Build a stream, compose stages, throttle to 100/sec, and materialize it as an IAsyncEnumerable that any .NET developer can consume.

// Build a composable stream pipeline
var results = Source
    .From(orders)
    .Via(Flow.Create<Order>()
        .Select(o => Validate(o))
        .Where(o => o.IsValid)
        .Throttle(100, TimeSpan.FromSeconds(1))) // rate limit: 100/sec
    .RunAsAsyncEnumerable(materializer);

// Consume as IAsyncEnumerable — familiar to any .NET dev
await foreach (var order in results)
{
    await ProcessOrder(order);
}

// KEY_CAPABILITIES

Stream Processing Features

// KAFKA_INTEGRATION

85% Less Code Than
Raw Confluent.Kafka

Akka.Streams.Kafka sits on top of Confluent.Kafka and eliminates the distributed systems boilerplate: built-in backpressure, supervision strategies for error handling, transparent partition rebalancing, and compositional design. Read the full comparison or explore the demo repo.

// CONNECTORS

// WHO_USES_THIS