Distributed Tracing Across .NET Services with OpenTelemetry: W3C Trace Context and Baggage

When a user clicks "checkout" on your e-commerce app, that single request might touch an API gateway, an order service, an inventory service, a payment service, and a notification service -- all within a few hundred milliseconds. When something goes wrong, or just goes slow, you need to be able to follow that request end-to-end. That's exactly what distributed tracing in .NET with OpenTelemetry is designed to provide.

Distributed tracing gives you a complete, visual picture of what happened across every service involved in handling a request. Every operation gets tagged with the same trace identifier. Load that trace in Jaeger, Grafana Tempo, or Azure Monitor and you see a timeline of every span -- who called whom, what succeeded, what failed, and where the latency was hiding. It turns a frustrating multi-service debugging session into a clear sequence of events.

In this article, you'll learn how distributed tracing works in .NET with OpenTelemetry. We'll cover the W3C Trace Context standard, automatic propagation through HttpClient and ASP.NET Core, Baggage for business-relevant metadata, manual propagation for message queues, and the sampling configuration that keeps multi-service traces intact from end to end.

If you're looking for the full picture, check out the OpenTelemetry in .NET: Complete Observability Guide.

What Is Distributed Tracing in .NET with OpenTelemetry?

A trace is a tree of spans -- each span has at most one parent, and the whole structure forms a directed acyclic graph with a single root. Each span represents a unit of work -- a database call, an HTTP request, a message queue operation. A root span starts the trace (often at the API gateway or load balancer), and every downstream operation creates child spans under it. Together, they form a hierarchical timeline of everything that happened during a single user request.

In a monolithic application, all of this happens in a single process. The trace context lives in memory as part of the Activity stack. That's relatively straightforward to reason about.

In a distributed system, context must cross process boundaries. Service A finishes a span, calls Service B over HTTP, and Service B needs to know it's part of the same trace -- not the start of a new one. Without some kind of propagation mechanism, Service B would create an unrelated trace, and you'd have no way to connect the two during an investigation.

This is where context propagation comes in. When Service A makes an outbound HTTP request, OpenTelemetry injects trace context into the HTTP headers. Service B reads those headers, creates its own spans as children of Service A's span, and the result is a single unified trace tree. One trace ID, multiple services, complete picture.

The important thing to understand: for common protocols like HTTP, you generally don't write this plumbing yourself. OpenTelemetry's instrumentation libraries handle it automatically. But understanding what's happening under the hood helps you debug broken traces and handle edge cases like message queues, gRPC streams, and custom protocols.

The W3C Trace Context Standard

The W3C Trace Context specification defines two HTTP headers: traceparent and tracestate. These are the default propagation mechanism in OpenTelemetry .NET, and they represent the industry-standard way to pass distributed trace identity across service boundaries.

Understanding the traceparent Header

The traceparent header carries the core trace identity. A typical value looks like this:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

It has four fields, separated by hyphens:

• Version (00): Always 00 in the current specification. Reserved for future versions of the standard.
• Trace ID (4bf92f3577b34da6a3ce929d0e0e4736): A 128-bit (32 hex character) identifier for the entire distributed trace. This value is identical across all services involved in a single user request -- it's the thread connecting every span.
• Parent Span ID (00f067aa0ba902b7): The 64-bit (16 hex character) identifier of the span in the calling service. Service B uses this as its root span's parent, correctly nesting it under Service A's span in the trace tree.
• Trace Flags (01): A single byte of feature flags. Currently only bit 0 is used: 01 means "sampled" (record this trace), 00 means "not sampled" (don't record it).

The trace-flags field is particularly important in distributed systems. It carries the sampling decision from the trace's origin service to every downstream service. We'll come back to why this matters when we cover sampling.

The tracestate Header

The tracestate header is optional. It carries vendor-specific or system-specific trace data alongside the traceparent. For example, a backend like Datadog might inject its own sampling priority into tracestate. Most setups don't need to interact with tracestate directly -- OpenTelemetry handles it transparently as part of context propagation.

Automatic Propagation with HttpClient and ASP.NET Core

The good news for HTTP-based service communication: you don't write any context propagation code yourself. OpenTelemetry's instrumentation libraries handle it entirely.

When you call AddHttpClientInstrumentation() in your tracing setup, OpenTelemetry registers a DelegatingHandler that automatically injects traceparent (and tracestate if present) into every outbound request. When you call AddAspNetCoreInstrumentation() in your receiving service, it automatically reads those headers from the incoming request and sets the parent context for all spans created during that request's handling.

Your business code stays clean. No manual header manipulation. No trace context threading. It just works.

// Service A (sends the request) -- automatically injects traceparent header
// With AddHttpClientInstrumentation, no extra code is needed
public class ServiceAClient
{
    private readonly HttpClient _httpClient;

    public ServiceAClient(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<string> GetDataFromServiceBAsync(string query)
    {
        // OpenTelemetry automatically adds:
        // traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
        // This links ServiceB's spans to ServiceA's current trace
        var response = await _httpClient.GetAsync($"/api/data?q={query}");
        return await response.Content.ReadAsStringAsync();
    }
}

// Service B (receives the request) -- automatically extracts traceparent
// AddAspNetCoreInstrumentation reads the traceparent header and sets the parent span
// No extra code needed in Service B either

One critical requirement: use IHttpClientFactory to create your HttpClient instances. OpenTelemetry's DelegatingHandler is registered through the factory pipeline. A manually created new HttpClient() bypasses that pipeline entirely -- the outbound request won't carry traceparent, and the receiving service will create a disconnected trace. This silent failure is one of the more common sources of broken distributed traces.

For a thorough look at correct HttpClient usage patterns, see the HttpClient in C# Complete Guide and the IHttpClientFactory in .NET guide.

How Context Flows Across Services

Let's trace exactly what happens when Service A calls Service B:

1. Service A receives an HTTP request. AddAspNetCoreInstrumentation creates a root span. Let's say it has trace ID abc123 and span ID span001.
2. Service A's business logic calls _httpClient.GetAsync(...).
3. OpenTelemetry's DelegatingHandler intercepts the outbound request. It reads the current Activity (trace ID abc123, span ID span001) and injects traceparent: 00-abc123-span001-01 into the request headers.
4. Service B receives the request. AddAspNetCoreInstrumentation reads the traceparent header and creates a new root span with the same trace ID abc123 and parent span ID span001.
5. Every subsequent span in Service B -- database calls, downstream HTTP calls, custom business spans -- is a child of Service B's root span.
6. Result: a single trace tree with abc123 as the trace ID, showing the complete call path from Service A through Service B.

When you load this trace in your observability backend, you see a unified timeline. Service A's handling, the network transit to Service B, and Service B's full processing chain are all visible in one view. Latency is attributable at every step. This is the core value proposition of distributed tracing.

Using Baggage for Business-Critical Context

Trace context (trace ID, span ID) is purely technical infrastructure. But sometimes you need to propagate business-relevant data alongside a request -- the tenant ID for a multi-tenant system, a correlation ID from a partner integration, or feature flags that affect how downstream services behave.

That's what Baggage is for. Baggage is a set of key-value pairs that travels with the trace context, automatically propagated to all downstream services via the baggage HTTP header. You write it in Service A and read it in Service B, C, or D without any additional plumbing.

using OpenTelemetry.Baggage;

// Service A -- add tenant ID to baggage before calling downstream
public async Task ProcessRequestAsync(string tenantId)
{
    // Set baggage -- this propagates to all downstream services
    Baggage.SetBaggage("tenant.id", tenantId);
    Baggage.SetBaggage("feature.flags", "new-checkout=true");

    await _downstreamClient.CallAsync();
}

// Service B or any downstream service -- read the baggage
public async Task HandleDownstreamAsync()
{
    // Baggage from Service A is available here automatically
    var tenantId = Baggage.GetBaggage("tenant.id");

    if (!string.IsNullOrEmpty(tenantId))
    {
        // Use the tenant ID for routing or data isolation
        _logger.LogInformation("Processing request for tenant {TenantId}", tenantId);
    }
}

A few important constraints to keep in mind when using Baggage:

Baggage is visible in headers. Any service, load balancer, or network device along the request path can read the baggage header. It's plaintext. Never put sensitive data in Baggage -- no passwords, tokens, PII, session keys, or payment data. Stick to non-sensitive identifiers and flags.

Baggage adds header overhead. Every key-value pair is serialized into the baggage header on every outbound HTTP request. Keep the payload small. Tenant IDs and feature flag names are appropriate. Serialized objects are not.

Baggage is not span tags. Tags are attached to a specific span and visible in your trace backend. Baggage is designed to influence downstream service behavior. They serve different purposes, but you can certainly copy baggage values into span tags if you want them surfaced in your trace visualization.

Manual Context Propagation for Message Queues

HTTP is the straightforward case -- OpenTelemetry handles it automatically. But what about RabbitMQ, Azure Service Bus, Kafka, or custom TCP protocols? These don't have a built-in HTTP header mechanism.

For these cases, you use Propagators.DefaultTextMapPropagator to manually inject and extract context. It works on any carrier -- a dictionary, message headers, or any key-value store you can provide adapters for.

using System.Diagnostics;
using OpenTelemetry;
using OpenTelemetry.Context.Propagation;

// Producer -- inject trace context into message metadata
public async Task PublishMessageAsync<T>(T message, string queueName)
{
    using var activity = _activitySource.StartActivity("PublishMessage");
    activity?.SetTag("messaging.system", "rabbitmq");
    activity?.SetTag("messaging.destination", queueName);

    var messageHeaders = new Dictionary<string, string>();

    // Inject trace context into message headers
    Propagators.DefaultTextMapPropagator.Inject(
        new PropagationContext(
            activity?.Context ?? default,
            Baggage.Current),
        messageHeaders,
        (dict, key, value) => dict[key] = value);

    await _queue.PublishAsync(new QueueMessage<T>
    {
        Body = message,
        Headers = messageHeaders,
        Queue = queueName
    });
}

// Consumer -- extract trace context from message metadata
public async Task ProcessMessageAsync<T>(QueueMessage<T> message)
{
    // Extract parent context from message headers
    var parentContext = Propagators.DefaultTextMapPropagator.Extract(
        default,
        message.Headers,
        (dict, key) => dict.TryGetValue(key, out var val) ? new[] { val } : Array.Empty<string>());

    using var activity = _activitySource.StartActivity(
        "ConsumeMessage",
        ActivityKind.Consumer,
        parentContext.ActivityContext);

    activity?.SetTag("messaging.system", "rabbitmq");

    await HandleMessageAsync(message.Body);
}

The pattern works on both sides: Inject serializes the current trace context and Baggage into whatever carrier you provide. Extract deserializes it back and returns a PropagationContext you use as the parent when starting your consumer span. The propagator handles the standard traceparent, tracestate, and baggage keys automatically.

This pattern applies to any asynchronous communication -- including scenarios where the message is processed hours after it was published. The trace still links the producer and consumer as part of the same logical operation, even across significant time gaps.

Sampling in Distributed Systems -- Why ParentBasedSampler Is Critical

Sampling in a single-service system is relatively simple: record some percentage of traces based on a rule (random rate, error status, etc.). In a distributed system, the picture gets more complicated -- and getting it wrong produces misleading partial traces.

Here's the problem with using TraceIdRatioBasedSampler alone in every service: each service makes an independent sampling decision. There's no guarantee that all services will decide the same way for a given trace. Service A might sample a trace, Service B might not. The result is a partial trace with some spans recorded and others missing -- which is arguably worse than no trace at all, because it gives you an incomplete and potentially misleading picture.

The solution is ParentBasedSampler. When a request arrives with a traceparent header, ParentBasedSampler reads the trace-flags field. If the calling service sampled the trace (01), this service also samples it. If the calling service didn't sample it (00), this service doesn't either. For requests with no parent (new traces entering the system), it falls back to whatever root sampler you configure -- typically TraceIdRatioBasedSampler.

// Service A -- head-based sampling
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .SetSampler(new ParentBasedSampler(
            // Sample 10% of new traces (no parent)
            new TraceIdRatioBasedSampler(0.1)))
        .AddOtlpExporter());

// Service B -- ALSO use ParentBasedSampler
// This is critical: if Service A sampled the trace (trace-flags = 01),
// ParentBasedSampler in Service B will ALSO sample it, keeping the trace complete.
// Without ParentBasedSampler, Service B might drop spans that Service A kept.
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .SetSampler(new ParentBasedSampler(
            new TraceIdRatioBasedSampler(0.1)))
        .AddOtlpExporter());

With this setup, Service A is the "head" of the sampling decision. It samples 10% of all new traces. Services B, C, and D downstream all respect that decision by reading the trace-flags field in the incoming traceparent header. You get complete, end-to-end traces for the sampled 10% rather than fragmented partial traces across a larger but inconsistent percentage.

This is why the trace-flags field in traceparent is more than an implementation detail -- it's the mechanism that makes consistent sampling across service boundaries possible.

Testing Distributed Tracing in .NET with OpenTelemetry

Automated tests that verify trace context is being propagated correctly are straightforward to write. The key is to start a real Activity in your test (which OpenTelemetry reads as the current span context) and then capture the outbound HTTP headers using a delegating handler mock.

[Fact]
public async Task OutboundRequest_PropagatesTraceContext()
{
    // Set up a test activity to simulate an incoming traced request
    using var testActivity = new Activity("TestOperation");
    testActivity.Start();

    // Your service calls downstream via HttpClient
    // Capture outbound headers using a mock handler
    HttpRequestMessage? capturedRequest = null;
    var mockHandler = new MockDelegatingHandler(request =>
    {
        capturedRequest = request;
        return new HttpResponseMessage(HttpStatusCode.OK);
    });

    var client = new HttpClient(mockHandler);
    await client.GetAsync("http://downstream/api/data");

    // Verify traceparent was injected
    capturedRequest!.Headers.Contains("traceparent").ShouldBeTrue();
    var traceparent = capturedRequest.Headers.GetValues("traceparent").First();
    traceparent.ShouldStartWith("00-");
    // The trace ID portion should match our test activity's trace ID
    traceparent.Contains(testActivity.TraceId.ToString()).ShouldBeTrue(
        "traceparent must contain the current trace ID");
}

MockDelegatingHandler is a test helper that captures the outgoing HttpRequestMessage and returns a configured response. The implementation registers with IHttpClientFactory's handler pipeline: services.AddHttpClient().AddHttpMessageHandler<MockDelegatingHandler>(). With a real Activity started in the test, OpenTelemetry reads it as the current span context and injects traceparent into outbound requests -- exactly as it would in production.

For end-to-end integration testing across full ASP.NET Core services, you can use WebApplicationFactory to spin up multiple service instances in-process and make real HTTP calls between them. This gives you high confidence that context propagation works correctly in a realistic environment. The Testing ASP.NET Core Web API guide covers the WebApplicationFactory foundation that makes this possible.

Common Pitfalls in Distributed Tracing .NET OpenTelemetry

Even with a well-understood standard, there are several recurring ways distributed tracing breaks in real codebases.

Using new HttpClient() directly. OpenTelemetry's propagation handler is registered through IHttpClientFactory. Span creation via the DiagnosticListener infrastructure works for all HttpClient instances -- the span will appear in your traces. However, W3C traceparent header injection (context propagation to downstream services) requires the DelegatingHandler pipeline, which only applies to clients created through IHttpClientFactory. A manually instantiated HttpClient creates spans but the outbound request will not carry traceparent, and downstream services create disconnected traces. This is a silent failure -- no errors, just missing context propagation. Use IHttpClientFactory consistently across the codebase.

Forgetting to register custom ActivitySources. If your business code creates spans with a custom ActivitySource named "MyApp.Checkout", you need .AddSource("MyApp.Checkout") or .AddSource("MyApp.*") in your tracing configuration. Without this registration, the spans are created but silently dropped -- they're never exported and never appear in traces.

Sampling mismatches between services. Using TraceIdRatioBasedSampler directly (without ParentBasedSampler) in multiple services causes the sampling math to diverge. Standardize on ParentBasedSampler with a consistent root sampler across your entire service fleet.

Losing Activity context across async boundaries. If you manually marshal work to a thread pool without flowing ExecutionContext, the Activity context won't be available on the worker thread. Normally async/await flows execution context correctly, but certain patterns (fire-and-forget Task.Run, manual ThreadPool.QueueUserWorkItem) can cause context loss. Be deliberate about where you start and stop activities relative to async boundaries.

Putting sensitive data in Baggage. The baggage header is plaintext and visible to any intermediary along the request path. Audit your baggage keys before deploying. Tenant IDs and feature flags: fine. User tokens or any PII: not appropriate for Baggage.

Modular Monolith vs Microservices -- The Same Mechanism at Different Scales

Distributed tracing is strongly associated with microservices, but it's equally valuable in a modular monolith. The underlying mechanism is identical -- Activity, ActivitySource, span parent-child relationships -- but in a modular monolith, context propagation happens in memory rather than over HTTP headers.

In a modular monolith, all modules run in the same process. Span context flows automatically through the in-memory Activity stack. You still get a complete trace tree showing which modules handled a request, how long each step took, and where errors occurred. The observability value is the same; the context transport is different.

When you eventually migrate from a monolith toward microservices, your observability code doesn't change. Modules that were previously in-process become separate services communicating over HTTP. OpenTelemetry handles context propagation automatically through traceparent headers. Your traces look structurally the same in your observability backend -- you just start seeing network latency where there was previously none.

This is one of the practical payoffs of adopting OpenTelemetry from the start in a modular monolith: the observability investment carries forward completely to a microservices architecture. The module communication patterns guide covers the communication approaches -- in-process events, direct calls, message contracts -- that determine how trace context flows between modules during the monolith phase.

The monolith vs microservices decision framework discusses observability as a dimension of architectural decisions. Distributed tracing complexity is a real cost of moving to microservices -- and having it handled by a standard framework like OpenTelemetry significantly reduces that cost.

Logging and Distributed Tracing Together

Distributed tracing and structured logging are complementary, not competing. When OpenTelemetry logging is configured alongside tracing, log records are automatically annotated with the current trace ID and span ID. This means you can navigate directly from a trace in Grafana Tempo to the correlated log entries in Grafana Loki (or any other OTel-compatible log backend).

In practice, this correlation is one of the most useful features of a complete OpenTelemetry setup. A trace tells you the shape and timing of what happened. Logs tell you the detail -- the exact error message, the specific value that triggered a branch, the user's session context. Together, they provide the complete picture that neither signal delivers alone.

The Logging in .NET guide covers structured logging patterns in depth, all of which pair naturally with OpenTelemetry's log correlation approach.

FAQ

What is distributed tracing .NET OpenTelemetry and why do I need it?

Distributed tracing .NET OpenTelemetry is a combination of the W3C Trace Context standard and the OpenTelemetry SDK that lets you correlate spans across multiple services into a single, unified trace. You need it when your application spans multiple processes or services and you want to understand the end-to-end flow of a request -- including where latency originates and which service produced an error. Without it, debugging multi-service systems means manually piecing together log entries from separate services, which is slow and error-prone.

What is the traceparent header and what do its fields mean?

The traceparent header carries trace identity across service boundaries. It has four hyphen-separated fields: version (always 00), trace ID (128-bit hex, identifies the whole trace), parent span ID (64-bit hex, identifies the calling span), and trace flags (1 byte, where 01 means sampled and 00 means not sampled). When Service B receives a request with traceparent, it uses the trace ID to link its spans to the same distributed trace, the parent span ID to correctly nest its root span under Service A's span, and the trace flags to inherit the sampling decision.

Why should I use ParentBasedSampler instead of just TraceIdRatioBasedSampler?

TraceIdRatioBasedSampler makes an independent sampling decision in each service. Because the ratio calculation uses only the trace ID, different services can mathematically arrive at different sampling decisions for the same trace. This produces partial traces -- some services record spans and others don't -- which are often worse than no trace at all. ParentBasedSampler reads the sampling decision from the incoming traceparent header's trace-flags field and respects it. Service A makes the head sampling decision; all downstream services follow it, producing complete traces for the sampled percentage.

What is Baggage and when should I use it?

Baggage is a set of key-value pairs propagated alongside trace context to all downstream services. It's appropriate for business-relevant metadata like tenant IDs, correlation IDs from external systems, and feature flag states. The key constraint: Baggage is transmitted in plaintext HTTP headers and is visible to any intermediary. Never put sensitive data (passwords, tokens, PII, payment data) in Baggage. Keep the payload small -- a few short string values are appropriate; serialized objects are not.

How do I propagate trace context through a message queue?

Message queues don't use HTTP headers, so you need to manually inject and extract context. Use Propagators.DefaultTextMapPropagator.Inject() on the producer side to serialize the current trace context and Baggage into the message's metadata dictionary. On the consumer side, use Propagators.DefaultTextMapPropagator.Extract() to deserialize the context and pass it as the parent ActivityContext when starting your consumer span. The full code pattern is shown in the "Manual Context Propagation for Message Queues" section above.

How do I test that my service is correctly propagating trace context?

Start a real Activity in your test to simulate an incoming traced request, then capture the outbound HTTP headers using a delegating handler mock. Assert that the traceparent header is present and that the trace ID portion matches the trace ID of your test activity. For end-to-end verification, use WebApplicationFactory to run multiple ASP.NET Core services in-process and make real HTTP calls between them. This gives you high-fidelity confidence that propagation works in a realistic environment.

Does distributed tracing work in a modular monolith or only with microservices?

Distributed tracing works well in both. In a modular monolith, all modules run in the same process and span context flows through the in-memory Activity stack -- no HTTP headers are needed. You still get full trace trees showing all module interactions, timing, and errors. In microservices, the same Activity mechanism is used, but context travels via traceparent headers over HTTP. The instrumentation code is identical in both cases, which means migrating from a modular monolith to microservices doesn't require rewriting your observability setup.

Disclaimer: Yes! This was written by AI based on input and direction from yours truly, Nick Cosentino, for Dev Leader. I have ensured that it represents my thoughts and perspectives on distributed tracing in .NET with OpenTelemetry.