Distributed tracing in .NET with OpenTelemetry

Navigating performance bottlenecks in modern microservices can feel like finding a needle in a haystack. This is where distributed tracing becomes a game-changer for .NET applications. By giving you a detailed view of how requests travel across different services, distributed tracing in .NET with OpenTelemetry simplifies debugging, enhances performance monitoring, and provides crucial visibility into complex cloud-based systems.

In this guide, we will explore how to implement distributed tracing in a .NET application using OpenTelemetry. You will learn about the unique advantages of .NET's native telemetry APIs, set up your first instrumented application, and discover best practices—such as using OTLP exporters—to future-proof your observability strategy.

Understanding distributed tracing and OpenTelemetry

Tracing involves monitoring and recording how an application executes a user’s request. Distributed tracing tracks this information across multiple interconnected services, allowing developers to analyze performance and trace failures back to their root cause.

OpenTelemetry is an open-source observability framework designed to standardize the collection, processing, and export of telemetry data (traces, metrics, and logs). The most important concepts include:

• Spans: Basic units of work in a trace, representing individual operations with metadata.
• Traces: Collections of spans showing the entire journey of a request through a system.
• Exporters: Components that send telemetry data to back-end systems (like an OTLP-compatible collector or Site24x7 APM Insight) for analysis.
• Processors: Tools that modify or enhance telemetry data before it’s exported.
• Instrumentations: Plugins that automatically capture telemetry data from frameworks and libraries.

Unlike many other languages where OpenTelemetry requires custom classes for tracing, .NET offers native support through the System.Diagnostics.Activity and System.Diagnostics.ActivitySource APIs. OpenTelemetry seamlessly integrates with these built-in classes, meaning that many standard .NET libraries are instrumented out-of-the-box without heavy external dependencies.

Prerequisites

To follow this tutorial, you need:

• A fundamental grasp of Microsoft .NET development.
• Visual Studio or Visual Studio Code for development (these instructions use Visual Studio Code).
• .NET SDK (version 7 or later).
• A terminal or command line interface.

How to set up the environment

Let’s set up a basic .NET web API project configured for distributed tracing.

First, open your terminal and run dotnet --version to verify that the .NET SDK is installed.

Next, create a new .NET web API by running the command dotnet new webapi -n DistributedTracingDemo. Navigate into your new project directory with cd DistributedTracingDemo and open it in VS Code using code ..

Now, install the required OpenTelemetry packages. In your terminal, run the following commands:

dotnet add package OpenTelemetry
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.Console
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.Http

This simple API will serve as our foundation. Let's create a simulated API that tracks air quality. Add a new folder named Models and create a file called AirQualityReport.cs:

namespace Examples.AspNetCore;  
public class AirQualityReport  
{ 
    public DateTime Date { get; set; }  
    public string? City { get; set; } 
    public int AQI { get; set; }
    public string Category => GetCategory(AQI);  

    private static string GetCategory(int aqi)  
    {  
       if (aqi <= 50) return "Good"; 
       if (aqi <= 100) return "Moderate"; 
       if (aqi <= 150) return "Unhealthy for Sensitive Groups"; 
       if (aqi <= 200) return "Unhealthy";  
       if (aqi <= 300) return "Very Unhealthy"; 
       return "Hazardous"; 
    }  
}  

Next, create a new controller inside the Controllers folder named AirQualityController.cs:

namespace Examples.AspNetCore.Controllers; 

using System; 
using System.Collections.Generic;  
using System.Linq;  
using Microsoft.AspNetCore.Mvc;  

[ApiController]  
[Route("[controller]")]  
public class AirQualityController : ControllerBase  
{  
    private static readonly string[] Cities = new[]  
    {  
        "New York", "Los Angeles", "Chicago", "Houston", "Phoenix" 
    };  

    [HttpGet]  
    public IEnumerable<AirQualityReport> Get()  
    {  
        var rng = new Random(); 
        return Enumerable.Range(1, 5).Select(index => new AirQualityReport 
        {  
            Date = DateTime.Now.AddDays(index),  
            City = Cities[rng.Next(Cities.Length)],  
            AQI = rng.Next(0, 500) 
        }).ToArray();  
    }  
}            

Integrating OpenTelemetry into the .NET application

We need to configure the OpenTelemetry SDK in Program.cs so it can listen for activities and export them appropriately.

Configure the resource

Defining a resource is crucial for identifying your traces. This configuration specifies the service name and instance details so you can correlate traces to the right application.

using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

var serviceName = "DistributedTracingDemo";
Action<ResourceBuilder> configureResource = r => r.AddService(
    serviceName: serviceName, 
    serviceVersion: "1.0.0", 
    serviceInstanceId: Environment.MachineName);            

Configure tracing and exporters

Add the OpenTelemetry services to the application’s service collection. We will set up instrumentations for inbound and outbound HTTP requests, specify a custom ActivitySource, and add our exporters.

builder.Services.AddOpenTelemetry()
    .ConfigureResource(configureResource)  
    .WithTracing(tracing => 
    {  
        tracing
            .AddSource("Examples.AspNetCore") // Listen for custom spans 
            .SetSampler(new AlwaysOnSampler())  
            .AddHttpClientInstrumentation() 
            .AddAspNetCoreInstrumentation();

        // Add the Console exporter for debugging
        tracing.AddConsoleExporter();
        
        // Best practice: Add the OTLP Exporter for production backends
        tracing.AddOtlpExporter(options => {
            options.Endpoint = new Uri("http://localhost:4317");
        });
    });            

While the Console Exporter is excellent for local debugging, utilizing the OpenTelemetry Protocol (OTLP) exporter is the industry standard for production. It allows you to seamlessly send your telemetry data to backends like Jaeger, Zipkin, or commercial APM solutions like Site24x7 APM Insight.

Implementing basic distributed tracing

In .NET, distributed tracing relies heavily on ActivitySource. Let's create an instrumentation class to manage custom traces.

Set up ActivitySource

Create an Instrumentation.cs file to encapsulate your custom metrics and tracing setup:

using System.Diagnostics;
using System.Diagnostics.Metrics;

public class Instrumentation: IDisposable 
{  
    internal const string ActivitySourceName = "Examples.AspNetCore"; 
    internal const string MeterName = "Examples.AspNetCore"; 
    private readonly Meter meter;  

    public Instrumentation() 
    {  
        string? version = typeof(Instrumentation).Assembly.GetName().Version?.ToString(); 
        this.ActivitySource = new ActivitySource(ActivitySourceName, version); 
        this.meter = new Meter(MeterName, version); 
    } 

    public ActivitySource ActivitySource { get; } 

    public void Dispose() 
    {  
        this.ActivitySource.Dispose();  
        this.meter.Dispose();  
    } 
}           

Inject instrumentation into services

In Program.cs, register the Instrumentation class as a singleton:

builder.Services.AddSingleton<Instrumentation>();           

Create custom spans

You can inject this class into your controllers to track specific business logic. In .NET, a span is represented by an Activity. Let's update the AirQualityController.cs:

private readonly ActivitySource _activitySource;

public AirQualityController(Instrumentation instrumentation)
{
    _activitySource = instrumentation.ActivitySource;
}

[HttpGet]  
public IEnumerable<AirQualityReport> Get()  
{  
    using var activity = _activitySource.StartActivity("GenerateAirQualityReports");
    activity?.SetTag("report.type", "daily_summary");
    
    // Existing logic here...
}            

Always use the null-conditional operator (activity?.SetTag) because StartActivity returns null if no listeners (like our OpenTelemetry SDK) are actively tracking the source.

Best practices for OpenTelemetry in .NET

To ensure your distributed tracing is robust and compatible across various tools, keep these best practices in mind:

• Use OTLP as your standard: Always prefer the OpenTelemetry Protocol (OTLP) exporter. It provides a vendor-agnostic way to transmit data efficiently and future-proofs your observability stack.
• Adopt semantic conventions: Follow the official OpenTelemetry Semantic Conventions for naming your tags and attributes (e.g., using http.method instead of custom names like HttpMethodName). This ensures tracing backends can parse and visualize the data correctly.
• Reuse ActivitySource: Do not create new instances of ActivitySource on every request. Make them static readonly or register them as Singletons via Dependency Injection to avoid unnecessary overhead.
• Consider zero-code instrumentation: For simpler applications or quick setups, explore zero-code instrumentation methods which can attach to your application without code modifications.

Testing and validating traces

Once you run your application using dotnet run, you can use the built-in Swagger UI to test and validate your trace data. Navigate to the Swagger endpoint in your browser and execute the Get method on the Air Quality API.

For validation, monitor your terminal console. Because we configured the Console Exporter, you should immediately see trace data printed for each HTTP request, including complete span IDs, parent trace IDs, and your custom tags.

If traces do not appear, ensure that your ActivitySource name matches exactly what was configured in the AddSource() method within your Program.cs.

Conclusion

Implementing distributed tracing in .NET with OpenTelemetry gives you an unparalleled view into the behavior and performance of your applications. By leveraging native APIs like System.Diagnostics.Activity, integrating standard OTLP exporters, and adhering to semantic conventions, you create a scalable, vendor-neutral observability pipeline.

With this foundation, you can easily connect your applications to powerful backends like Site24x7 APM Insight to analyze traces, spot bottlenecks, and dramatically improve your system's reliability.