BrandGhost
OpenTelemetry Traces .NET -- ActivitySource, Spans, and the Tracing Pipeline

OpenTelemetry Traces .NET -- ActivitySource, Spans, and the Tracing Pipeline

If you have ever stared at logs trying to piece together a slow request across multiple services, you already understand why distributed traces in .NET applications are so valuable. Setting up OpenTelemetry traces .NET applications use gives you visibility that logs alone cannot provide -- not just what happened, but where, when, and how long each piece took. This article walks through the full picture: what traces and spans actually are, how .NET implements them natively through System.Diagnostics.Activity, and how to wire up the OpenTelemetry tracing pipeline so your data reaches a backend you can query.

Tracing is one of the three pillars of observability alongside logs and metrics. It is the signal that gives you a call tree -- a structured view of your request's journey through your system. Let's dig in, from first principles through a working end-to-end example.

If you're looking for the full picture, check out the OpenTelemetry in .NET: Complete Observability Guide.

What Are Traces and Spans?

A trace represents the complete journey of a single request through your system. Every trace has a globally unique trace ID -- a 128-bit random value that links all the work triggered by that request, no matter how many services or processes were involved.

Within a trace, work is organized as spans. Each span represents one unit of work: handling an HTTP request, executing a database query, calling an external API, publishing a message. A span has a span ID, a start timestamp, an end timestamp, and a status. It may also have a parent span ID, which is what creates the parent-child nesting you see in waterfall views in tools like Jaeger or Honeycomb.

A typical trace might look like this:

  • Root span: HTTP POST /orders (50ms total)
    • Child span: ValidateOrder (5ms)
    • Child span: INSERT orders (12ms)
    • Child span: HTTP POST notification-service/send (28ms)

The root span starts when a request arrives at your service. Child spans are created for each sub-operation. When all children complete and the root span ends, the full trace is assembled in your observability backend.

W3C Trace Context is the standard that carries this context across service boundaries. Two HTTP headers -- traceparent and tracestate -- encode the trace ID, parent span ID, and trace flags. When your service makes an outbound HTTP call with these headers set, the downstream service reads them and creates its spans as children of the span that made the call. This is what makes distributed tracing actually distributed.

How .NET Implements Tracing -- System.Diagnostics.Activity

Here is something that surprises many .NET developers: you do not need the OpenTelemetry SDK to create trace data. Activity has been part of System.Diagnostics since .NET Core 2.0. ActivitySource -- the factory for creating Activity instances with the listener-based architecture that OpenTelemetry hooks into -- was introduced in .NET 5.

Activity is the .NET representation of a span. ActivitySource is the factory that creates Activity instances. OpenTelemetry does not replace these types -- it listens to them. When the SDK is configured, it hooks into ActivitySource and exports the data it captures.

This separation is intentional. If you instrument a class library using ActivitySource, applications that host your library can choose to wire up OpenTelemetry (or any other listener) without any changes to your library code. The Dependency Inversion Principle is at work here: your library depends on an abstraction (ActivitySource), not a concrete exporter. This is the same principle that makes the rest of well-structured .NET code testable and extensible.

An ActivitySource is typically a static field on the class that owns it:

using OpenTelemetry.Trace;
using System.Diagnostics;

namespace MyApp.Services;

public class OrderService
{
    // One ActivitySource per component -- static, named, versioned
    private static readonly ActivitySource _activitySource =
        new("MyApp.OrderService", "1.0.0");

    public async Task<Order> ProcessOrderAsync(int orderId)
    {
        // StartActivity returns Activity? -- null when no listener is registered
        using var activity = _activitySource.StartActivity("ProcessOrder");
        activity?.SetTag("order.id", orderId);

        // ... business logic ...

        return await GetOrderAsync(orderId);
    }
}

A few critical details here. The source is static -- you create one per component, not one per request or method call. The name you choose is the identifier you later register with the SDK. StartActivity returns Activity? because if no listener is registered, it returns null with near-zero overhead. The null-conditional ?. handles this gracefully. The using declaration ensures the activity is disposed (and ended) when the method returns, even on exception -- that disposal is what records the end timestamp.

Setting Up OpenTelemetry Traces .NET Pipeline

To make OpenTelemetry traces .NET applications emit actually flow somewhere, you need to register a listener via the OpenTelemetry SDK. Add these NuGet packages to your project:

<PackageReference Include="OpenTelemetry" Version="1.9.0" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.9.0" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.9.0" />
<PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.9.0" />

Then configure the pipeline in Program.cs:

using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(
            serviceName: "MyApp",
            serviceVersion: "1.0.0"))
    .WithTracing(tracing => tracing
        .AddSource("MyApp.OrderService")      // register your ActivitySource
        .AddAspNetCoreInstrumentation()        // auto-instrument incoming HTTP
        .AddHttpClientInstrumentation()        // auto-instrument outgoing HTTP
        .AddConsoleExporter());                // swap for AddOtlpExporter() in production

var app = builder.Build();
// ... rest of startup

The AddSource("MyApp.OrderService") call is what activates your ActivitySource. Without it, every StartActivity() call returns null and no spans are captured. The name must match the string you passed to the ActivitySource constructor.

ConfigureResource with AddService sets the service.name attribute on all exported spans -- this is how your observability backend knows which service produced a trace. Missing this is one of the most common pitfalls; traces without a service name are hard to filter and query.

This setup fits naturally into any ASP.NET Core Web API startup. It follows the same builder-pattern DI registration you already use for everything else in your application.

Auto-Instrumentation vs Manual Instrumentation

OpenTelemetry for .NET gives you two complementary approaches to creating spans, and understanding when to use each is important.

Auto-instrumentation means the SDK generates spans automatically for common operations without you writing any tracing code:

  • AddAspNetCoreInstrumentation() creates a span for every incoming HTTP request, populated with route, method, and status code
  • AddHttpClientInstrumentation() creates a span for every HttpClient call and injects W3C trace context headers automatically
  • AddSqlClientInstrumentation() captures database query spans
  • AddEntityFrameworkCoreInstrumentation() wraps EF Core queries

These are the spans that tell you about the edges of your system -- requests coming in, requests going out, database calls. You get them for free.

Manual instrumentation means you create spans explicitly in your own business logic. Use this for operations the SDK cannot see: processing a batch, executing a business rule, running a background job, calling an internal service layer. This is where ActivitySource.StartActivity() comes in.

The two approaches layer together cleanly. A single trace might have an auto-instrumented root span from AddAspNetCoreInstrumentation, then child spans you created manually inside your service layer. The automatic span handles the HTTP boundary; your manual spans explain what happened inside. This combination is how production OpenTelemetry traces .NET services typically look in practice.

Creating Parent/Child Spans with Tags, Events, and Exceptions

The key to useful traces is not just creating spans -- it is enriching them with context. Here is a fuller example showing tags, events, and exception recording:

public async Task<bool> ValidateAndChargeAsync(int orderId, decimal amount)
{
    using var activity = _activitySource.StartActivity(
        "ValidateAndCharge",
        ActivityKind.Internal);

    activity?.SetTag("order.id", orderId);
    activity?.SetTag("charge.amount", amount);

    try
    {
        // ValidateOrder becomes a child span automatically -- Activity.Current flows
        using var validateActivity = _activitySource.StartActivity("ValidateOrder");
        var isValid = await ValidateOrderAsync(orderId);

        if (!isValid)
        {
            activity?.SetStatus(ActivityStatusCode.Error, "Order validation failed");
            return false;
        }

        activity?.AddEvent(new ActivityEvent("ValidationPassed"));

        using var chargeActivity = _activitySource.StartActivity("ChargeCustomer");
        chargeActivity?.SetTag("payment.provider", "Stripe");
        await ChargeAsync(amount);

        activity?.SetStatus(ActivityStatusCode.Ok);
        return true;
    }
    catch (Exception ex)
    {
        activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
        activity?.RecordException(ex);   // records type, message, and stack trace
        throw;
    }
}

ValidateOrder and ChargeCustomer are child spans of ValidateAndCharge because Activity.Current was set to the parent when they were created. This flows automatically through async/await via AsyncLocal<T> -- you do not need to pass activities as method parameters.

Tags (attributes) are key-value descriptors of the operation. Use OpenTelemetry semantic conventions like http.method, db.system, and messaging.system where they apply -- this makes your data compatible with standard dashboards and alert rules.

Events are timestamped annotations that mark something happening during a span, distinct from the span's own metadata. Useful for retry attempts, cache hits, progress milestones.

RecordException adds the exception as a structured event with type, message, and stack trace. Combined with SetStatus(Error), this surfaces the span in error-rate views and makes exceptions queryable in your backend.

A Complete Service-to-Service Tracing Example

Here is a fuller example showing how tracing spans flow across a service boundary using HttpClient. This demonstrates the full OpenTelemetry traces .NET pipeline with W3C context propagation:

// OrderService -- the upstream caller
public class OrderService
{
    private static readonly ActivitySource _activitySource =
        new("MyApp.OrderService", "1.0.0");

    private readonly HttpClient _httpClient;

    public OrderService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<OrderResult> PlaceOrderAsync(OrderRequest request)
    {
        using var activity = _activitySource.StartActivity(
            "PlaceOrder",
            ActivityKind.Internal);

        activity?.SetTag("order.customer_id", request.CustomerId);
        activity?.SetTag("order.item_count", request.Items.Count);

        try
        {
            // HttpClientInstrumentation automatically injects traceparent header here
            // The downstream service reads it and creates child spans under this trace
            var response = await _httpClient.PostAsJsonAsync(
                "/api/inventory/reserve",
                request.Items);

            response.EnsureSuccessStatusCode();

            activity?.SetTag("inventory.reserved", true);
            activity?.SetStatus(ActivityStatusCode.Ok);

            return new OrderResult(Success: true);
        }
        catch (HttpRequestException ex)
        {
            activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
            activity?.RecordException(ex);
            throw;
        }
    }
}

// InventoryService -- the downstream receiver
// Program.cs for InventoryService:
builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService("MyApp.Inventory", "1.0.0"))
    .WithTracing(t => t
        .AddSource("MyApp.InventoryService")
        .AddAspNetCoreInstrumentation()  // reads traceparent header, creates child span
        .AddOtlpExporter());

When OrderService makes the HTTP call, AddHttpClientInstrumentation injects traceparent into the request headers. When InventoryService receives it, AddAspNetCoreInstrumentation reads those headers and creates the incoming span as a child of the remote parent. Both services emit spans with the same trace ID -- they appear as one connected trace in your observability backend. This is the promise of distributed systems observability: understanding service dependencies without guessing.

Common Pitfalls with OpenTelemetry Traces .NET Applications Hit

A few mistakes show up repeatedly when teams add OpenTelemetry traces to .NET applications.

Not disposing activities. If you create an Activity with StartActivity() but never dispose it, the span never ends. Your trace will have a phantom open span that never appears in your backend. The fix is always a using declaration or explicit Dispose() call, even on the error path. The try/catch in the examples above is deliberate -- the using wraps the whole block.

Missing service.name. Without ConfigureResource(r => r.AddService(...)), your spans are exported without a service name attribute. Most observability backends will accept them, but filtering by service becomes impossible and the UI often shows these as unnamed or unknown services. This is one of the first things to check when traces look broken in your backend.

Forgetting AddSource(). If StartActivity() always returns null in production but works in tests, check whether your source name is registered. The source name in AddSource() must match the string passed to new ActivitySource(...) exactly -- case-sensitive.

Sampling blindspots. By default, the SDK uses a ParentBased sampler that defers to the incoming trace context. In development this is usually fine. In production at high traffic, you often want head-based sampling to reduce volume. If sampling is too aggressive, you lose traces for rare events. Understand your sampling configuration before shipping to production.

Over-instrumenting. Adding a manual span around every private method creates noise rather than signal. Spans should represent operations that have meaningful duration and context. A three-line helper method that runs in under a microsecond does not need a span.

Correlating Traces with Logs

Tracing becomes more powerful when correlated with your structured logs. Modern .NET logging automatically links with the current trace -- when using the OpenTelemetry logging integration, TraceId and SpanId from Activity.Current are injected into each log scope.

If you use Serilog enrichers, you can enrich every log entry with the current trace and span IDs automatically. In your observability backend, you can jump from a log entry to its corresponding trace span and see the full request context -- which is exactly the kind of correlation that turns log debugging from guesswork into navigation. The complete logging guide for .NET covers the OpenTelemetry logging integration in detail.

Similarly, ASP.NET Core middleware can enrich the current span with request-level context -- user ID, tenant ID, correlation headers -- so every child span in the request inherits that context. The middleware enriches Activity.Current, and that data appears on all downstream spans without any changes to your business logic.

Frequently Asked Questions

The OpenTelemetry traces .NET ecosystem has a few concepts that trip up developers when they are getting started. Here are the questions that come up most often.

What is the difference between a Trace and a Span in OpenTelemetry traces .NET?

A trace is the complete picture of one request -- all the work it triggered, across all services and processes. A span is a single unit of work within that trace. Traces are made up of spans. Every span belongs to exactly one trace (identified by the trace ID) and may have a parent span that it is nested under.

Why does ActivitySource.StartActivity() return null?

StartActivity() returns null when no listener is subscribed to that source. This is by design -- it means uninstrumented code has near-zero overhead. The OpenTelemetry SDK subscribes via AddSource() in your WithTracing() configuration. If your source name does not match what you registered with AddSource(), all StartActivity() calls return null silently. Always use null-conditional operators (?.) when accessing the returned Activity?.

How does Activity.Current flow correctly through async/await?

Activity.Current is backed by AsyncLocal<Activity?>, which means it is captured at each await boundary and restored on continuation. When you await a method, the calling context -- including Activity.Current -- flows into the called method. Child spans created in nested methods automatically become children of the ambient parent without any parameter threading. This is what makes the tracing model non-invasive to your method signatures.

When should I use auto-instrumentation vs manual instrumentation for OpenTelemetry tracing in .NET?

Auto-instrumentation (via AddAspNetCoreInstrumentation, AddHttpClientInstrumentation, etc.) covers the system-level operations at service boundaries -- incoming requests, outgoing HTTP calls, database queries. Use it for everything it covers. Manual instrumentation adds context inside your service -- business operations, processing steps, significant decision points. The two work together: auto-instrumentation gives you the frame; manual instrumentation fills in what happened inside.

How do I propagate OpenTelemetry trace context across HTTP service calls?

AddHttpClientInstrumentation() automatically injects W3C Trace Context headers (traceparent, tracestate) into outgoing HttpClient requests. The downstream service's AddAspNetCoreInstrumentation() reads those headers on the incoming request and creates the root span for that service as a child of the remote parent span. The trace ID is shared across both services, so both services' spans appear as one connected trace in your backend.

Should I use a static ActivitySource or inject it via dependency injection?

The standard pattern is a static readonly field per component. ActivitySource is lightweight and its identity is its name, not its instance. Library code that uses a static source does not need to take any dependency on the OpenTelemetry SDK or your application's DI container. If you need the service name or version to come from configuration at runtime, wrap the source in a small singleton service registered in your ASP.NET Core Web API DI container.

How do I test that my OpenTelemetry traces are correct?

You can capture activities in unit and integration tests using ActivityListener without running any OpenTelemetry infrastructure:

var recorded = new List<Activity>();
var listener = new ActivityListener
{
    ShouldListenTo = src => src.Name == "MyApp.OrderService",
    Sample = (ref ActivityCreationOptions<ActivityContext> _) =>
        ActivitySamplingResult.AllData,
    ActivityStopped = a => recorded.Add(a)
};
ActivitySource.AddActivityListener(listener);

// Run your code, then assert on `recorded`

This is a solid pattern for integration tests that verify instrumentation is correct and will not regress silently. Tag names and operation names are part of your observability contract -- test them.

Wrapping Up

OpenTelemetry traces .NET are built on a clean foundation. System.Diagnostics.Activity and ActivitySource give you a portable, SDK-independent tracing API. The OpenTelemetry SDK wires up listeners and exporters through a few lines of startup code. Auto-instrumentation handles the system boundaries; manual instrumentation fills in your business logic. And Activity.Current's ambient context model keeps tracing non-invasive to your method signatures.

The core takeaways for getting OpenTelemetry traces .NET right: use a static readonly ActivitySource per component, null-check your Activity? references, register sources with AddSource(), set service.name via ConfigureResource, and follow semantic conventions for your tag names. With those foundations in place, your traces will be useful -- whether you are debugging a slow request, tracing an error across service boundaries, or understanding how error handling propagates through your distributed system.

Start with one service. Export to the console. See the spans appear. Then expand outward from there.


Disclaimer: Yes! This was written by AI based on input and direction from yours truly, Nick Cosentino, for Dev Leader. I have ensured that it represents my thoughts and perspectives on OpenTelemetry tracing in .NET.

OpenTelemetry in .NET: Complete Observability Guide

The complete OpenTelemetry .NET guide for developers: traces, metrics, and logs with C# examples, ASP.NET Core setup, exporters, and distributed tracing.

OpenTelemetry and Observability in Microsoft Agent Framework

Set up microsoft agent framework opentelemetry observability in C# to track token usage, latency, tool calls, and errors in your AI agents.

HttpClient Logging and Observability in .NET 10: ILogger, DelegatingHandler, and OpenTelemetry

How to implement HttpClient logging and observability in .NET 10 -- covering built-in ILogger integration, DelegatingHandler for custom request logging, OpenTelemetry tracing and metrics, and redacting sensitive headers in production.

An error has occurred. This application may no longer respond until reloaded. Reload