Using Benchmarking in C# (BenchmarkDotNet)

Learn benchmarking in C# using BenchmarkDotNet to measure performance and optimize your code with accurate metrics.

Performance measurement (benchmarking) is the comparison of two or more approaches in terms of speed, memory consumption, and scalability. In the .NET world, BenchmarkDotNet is the industry standard for this task. It optimizes compilation, performs warmups, runs multiple iterations, produces statistical analyses (mean, median, deviation), and generates detailed reports. In this article, you’ll see how to install BenchmarkDotNet, use it for basic benchmarking, and apply it effectively in real-world scenarios.

Installation

Add the BenchmarkDotNet package to your test/benchmark project:


dotnet new console -n BenchmarkExample
cd BenchmarkExample
dotnet add package BenchmarkDotNet

It is recommended to run your program in Release mode; BenchmarkDotNet handles this automatically, but your project configuration should be up to date.

First Benchmark: Simple Comparison

The following example compares two string concatenation methods. Each benchmark method is marked with [Benchmark]; the runner is executed via BenchmarkRunner.Run<T>().


using System;
using System.Text;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

public class StringBenchmarks
{
    private const int N = 1000;

    [Benchmark]
    public string PlusOperator()
    {
        string s = string.Empty;
        for (int i = 0; i < N; i++)
            s += "x";
        return s;
    }

    [Benchmark]
    public string StringBuilderAppend()
    {
        var sb = new StringBuilder();
        for (int i = 0; i < N; i++)
            sb.Append('x');
        return sb.ToString();
    }
}

class Program
{
    static void Main() => BenchmarkRunner.Run<StringBenchmarks>();
}

When executed, it generates detailed reports (.md, .csv, .html) under the BenchmarkDotNet.Artifacts directory.

Key Attributes

[Benchmark]: Marks the method to be measured.
[GlobalSetup] / [GlobalCleanup]: Runs once before/after all benchmarks (e.g., data preparation).
[IterationSetup] / [IterationCleanup]: Runs before/after each iteration.
[Params(...)]: Executes the same benchmark with different parameter values.
[MemoryDiagnoser]: Adds memory metrics (Allocated B, Gen0/1/2).


using BenchmarkDotNet.Attributes;

[MemoryDiagnoser]
public class SearchBench
{
    private int[] _data;

    [Params(1_000, 100_000)]
    public int Size;

    [GlobalSetup]
    public void Setup()
    {
        var rnd = new Random(42);
        _data = new int[Size];
        for (int i = 0; i < Size; i++) _data[i] = rnd.Next();
        Array.Sort(_data);
    }

    [Benchmark(Baseline = true)]
    public bool LinearSearch() => Array.Exists(_data, x => x == 42);

    [Benchmark]
    public bool BinarySearch() => Array.BinarySearch(_data, 42) >= 0;
}

The option Baseline = true allows the results table to include a ratio column for comparison.

Jobs, Runtime, and Environment Settings

With Job configurations, you can specify runtime targets, JIT compiler, platform, GC mode, and more.


using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;

[SimpleJob(RuntimeMoniker.Net80)]
[SimpleJob(RuntimeMoniker.Net472)]
[MemoryDiagnoser]
public class HashBench
{
    [Benchmark] public int DefaultHash() => "abcdef".GetHashCode();
}

class Program
{
    static void Main() => BenchmarkRunner.Run<HashBench>();
}

This configuration allows you to compare performance across multiple runtimes (e.g., .NET 8.0 vs .NET Framework 4.7.2).

Warmup, Iteration, and Outlier Management

BenchmarkDotNet automatically performs warmups and multiple measurement iterations to reduce noise. You can also configure these manually:


using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;

[Config(typeof(MyConfig))]
public class MathBench
{
    private class MyConfig : ManualConfig
    {
        public MyConfig()
        {
            AddColumn(BenchmarkDotNet.Columns.StatisticColumn.Mean,
                      BenchmarkDotNet.Columns.StatisticColumn.StdDev);
            AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);
            AddJob(BenchmarkDotNet.Jobs.Job.Default
                .WithWarmupCount(3)
                .WithIterationCount(10));
        }
    }

    [Benchmark] public double Sqrt() => Math.Sqrt(12345.6789);
}

The resulting statistics include Mean, Median, StdDev, Min, and Max values; outliers can be filtered for cleaner analysis.

Parametric Comparisons

With [Params], you can compare different data sizes or strategies within a single table.


using BenchmarkDotNet.Attributes;
using System.Linq;

[MemoryDiagnoser]
public class SumBench
{
    [Params(10, 1000, 100000)]
    public int N;

    private int[] _arr;

    [GlobalSetup]
    public void Setup() => _arr = Enumerable.Range(1, N).ToArray();

    [Benchmark(Baseline = true)]
    public long ForLoop()
    {
        long s = 0;
        for (int i = 0; i < _arr.Length; i++) s += _arr[i];
        return s;
    }

    [Benchmark]
    public long LinqSum() => _arr.Sum(x => (long)x);
}

The results allow you to observe how performance changes with different values of N — helping distinguish between micro-optimizations and algorithmic differences.

Advanced Diagnostics: Disassembly and Hardware Counters

DisassemblyDiagnoser: Displays the assembly generated by the JIT compiler (for advanced analysis).
HardwareCounters: Provides CPU counters (CacheMisses, BranchMispredictions, etc. — depends on platform support).


using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Diagnosers;

[DisassemblyDiagnoser(printSource: true, maxDepth: 2)]
public class CryptoBench
{
    [Benchmark] public byte[] GuidNew() => Guid.NewGuid().ToByteArray();
}

These diagnostics are highly useful for detecting unnecessary allocations and instruction-level differences in hot code paths.

Exporting Results (Exporters)

Results can be exported as Markdown, CSV, or HTML — ideal for CI/CD reporting.


using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Exporters;
using BenchmarkDotNet.Loggers;

public class CsvConfig : ManualConfig
{
    public CsvConfig()
    {
        AddExporter(CsvExporter.Default);
        AddLogger(ConsoleLogger.Default);
    }
}

Common Mistakes and Best Practices

Don’t compare with Stopwatch: Microbenchmarks have high noise; JIT and GC effects can distort results.
Use Release + x64: Debug builds are misleading — JIT optimizations are disabled.
Avoid dead code elimination: If the benchmark result isn’t used, JIT may remove it. Return the result or use Consumer.
Warmup is essential: Initial runs are affected by JIT and cache; BenchmarkDotNet handles this automatically.
Use realistic datasets: Artificially small data can lead to incorrect conclusions.
Run on a dedicated, consistent machine: Background tasks or antivirus processes can cause noise.

Real-World Example: JSON Serialization Comparison

The example below compares JSON serialization across different libraries (for illustration) and measures memory allocation. You can clearly see which library performs better and allocates less for your data model.


using System;
using System.Collections.Generic;
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
// Extra: If using Newtonsoft.Json, add the package and uncomment the line below
// using Newtonsoft.Json;

[MemoryDiagnoser]
public class JsonBench
{
    private List<Person> _list;

    [Params(100, 10_000)]
    public int Count;

    [GlobalSetup]
    public void Setup()
    {
        _list = new List<Person>(Count);
        for (int i = 0; i < Count; i++)
            _list.Add(new Person { FirstName = "Ada", LastName = "Lovelace", Age = 28, Active = (i % 2) == 0 });
    }

    [Benchmark(Baseline = true)]
    public string SystemTextJson() => JsonSerializer.Serialize(_list);

    // [Benchmark]
    // public string NewtonsoftJson() => JsonConvert.SerializeObject(_list);
}

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int Age { get; set; }
    public bool Active { get; set; }
}

class Program
{
    static void Main() => BenchmarkRunner.Run<JsonBench>();
}

The report displays metrics such as Mean, Error, StdDev, and Allocated for each method. A lower Allocated value reduces GC pressure and makes a significant difference under heavy workloads.

CI/CD Integration and Repeatability

Run benchmarks as a separate pipeline stage; archive results as .csv or .md.
Use consistent hardware or VM; set CPU power mode to “High Performance.”
Mark baselines in comparative PRs to detect performance regressions.

TL;DR

BenchmarkDotNet is the standard for reliable, repeatable .NET performance measurements.
[Benchmark], [Params], and [MemoryDiagnoser] are the key attributes.
Always benchmark in Release/x64 mode with realistic data — ensure the result is used.
Use Jobs to compare multiple runtimes or GC configurations.
Export reports as CSV/MD/HTML for CI/CD tracking and catch regressions with baseline benchmarks.