When to Use Iterator Pattern in C#: Decision Guide with Examples

You've got foreach loops everywhere in your C# code. Lists, arrays, dictionaries -- they all support enumeration out of the box. So why would you ever need to build a custom iterator? The answer is that built-in collections cover the common case, but when to use the iterator pattern in C# becomes a real question once your data doesn't fit neatly into a List<T>. Maybe you're reading millions of lines from a file. Maybe you're walking a tree structure. Maybe you're pulling paginated results from an API and want to hide that complexity behind a simple loop.

This article gives you a structured decision framework for recognizing when the iterator pattern earns its place in your code versus when you should lean on what .NET gives you for free. We'll walk through decision criteria, three practical scenarios with C# code, and the situations where a custom iterator adds unnecessary complexity. If you're building up your design pattern toolkit, consider checking out the strategy design pattern for another pattern that helps you swap out behavior cleanly.

Decision Criteria: When Custom Iterators Add Value

Not every data source maps to a List<T> or an array. And not every traversal is a simple front-to-back scan. The question of when to use the iterator pattern in C# boils down to a few clear signals.

Custom Data Structures That Don't Map to Standard Collections

If you've built a tree, a graph, a skip list, or any data structure that isn't a flat sequence, the built-in collection types won't expose it the way consumers expect. A custom iterator lets you present your data structure as an IEnumerable<T> so callers can use foreach without understanding the internal layout. This is especially valuable when your data structure is part of a composite design pattern where the hierarchy can be arbitrarily deep.

Lazy Evaluation of Expensive Computations

When each element in a sequence is expensive to compute -- think parsing, transforming, or generating -- materializing the entire collection upfront wastes time and memory if the consumer only needs a few elements. The iterator pattern in C# with yield return lets you compute elements one at a time, on demand. The consumer pulls what it needs, and the rest never gets computed.

Streaming Data from External Sources

Files, network streams, database cursors, and API endpoints aren't collections sitting in memory. They're sources you read from incrementally. A custom iterator wraps the read-ahead logic so callers see a clean foreach loop. This is the same principle behind how IAsyncEnumerable<T> works for async data sources in .NET.

Beyond these three signals, there's one more worth considering.

Multiple Traversal Strategies Over the Same Data

A binary tree can be traversed in-order, pre-order, or post-order. A graph can be traversed depth-first or breadth-first. If consumers of your data structure need different traversal orders, the iterator pattern lets you expose multiple IEnumerable<T> methods -- one per traversal strategy -- without forcing the consumer to understand the traversal algorithm. This pairs naturally with the strategy design pattern, where the traversal algorithm itself is interchangeable.

Scenario: Lazy File Processing

One of the clearest demonstrations of when to use the iterator pattern in C# is processing large files. If you call File.ReadAllLines(), you load the entire file into memory as a string array. For a file with millions of lines, that's a significant allocation. An iterator reads one line at a time.

using System.Collections.Generic;
using System.IO;

public static class LazyFileReader
{
    // Reads lines one at a time using yield return
    public static IEnumerable<string> ReadLines(
        string filePath)
    {
        using var reader = new StreamReader(filePath);

        while (!reader.EndOfStream)
        {
            yield return reader.ReadLine()!;
        }
    }
}

The yield return keyword is the key. Each time the consumer's foreach asks for the next element, execution resumes inside ReadLines, reads one line, and suspends again. The StreamReader stays open across iterations, and the using statement ensures it gets disposed when enumeration finishes or the consumer breaks out early.

Here's how you'd use it:

// Only one line is in memory at any given time
foreach (var line in LazyFileReader.ReadLines("server-log.txt"))
{
    if (line.Contains("ERROR"))
    {
        Console.WriteLine(line);
    }
}

Compare this to the eager approach:

// Loads the ENTIRE file into memory at once
var allLines = File.ReadAllLines("server-log.txt");

foreach (var line in allLines)
{
    if (line.Contains("ERROR"))
    {
        Console.WriteLine(line);
    }
}

With a 2 GB log file containing millions of lines, ReadAllLines allocates a massive string array plus the string data itself. The lazy iterator holds exactly one string in flight at a time. If you're only looking for error lines and breaking early, the iterator may read just a fraction of the file.

Note that .NET does include File.ReadLines() (not ReadAllLines), which uses this exact pattern internally. That's how fundamental the iterator pattern is to the .NET ecosystem -- the framework itself uses it to solve this problem.

Scenario: Tree Traversal

Trees are a natural fit for the iterator pattern in C# because they're not flat sequences. A consumer shouldn't need to understand recursion or maintain a stack just to visit every node. Multiple traversal strategies make this even more compelling.

using System.Collections.Generic;

public sealed class BinaryTreeNode<T>
{
    public T Value { get; }
    public BinaryTreeNode<T>? Left { get; }
    public BinaryTreeNode<T>? Right { get; }

    public BinaryTreeNode(
        T value,
        BinaryTreeNode<T>? left = null,
        BinaryTreeNode<T>? right = null)
    {
        Value = value;
        Left = left;
        Right = right;
    }

    // In-order: Left, Root, Right
    public IEnumerable<T> InOrder()
    {
        if (Left is not null)
        {
            foreach (var item in Left.InOrder())
            {
                yield return item;
            }
        }

        yield return Value;

        if (Right is not null)
        {
            foreach (var item in Right.InOrder())
            {
                yield return item;
            }
        }
    }

    // Pre-order: Root, Left, Right
    public IEnumerable<T> PreOrder()
    {
        yield return Value;

        if (Left is not null)
        {
            foreach (var item in Left.PreOrder())
            {
                yield return item;
            }
        }

        if (Right is not null)
        {
            foreach (var item in Right.PreOrder())
            {
                yield return item;
            }
        }
    }

    // Post-order: Left, Right, Root
    public IEnumerable<T> PostOrder()
    {
        if (Left is not null)
        {
            foreach (var item in Left.PostOrder())
            {
                yield return item;
            }
        }

        if (Right is not null)
        {
            foreach (var item in Right.PostOrder())
            {
                yield return item;
            }
        }

        yield return Value;
    }
}

Each traversal method returns IEnumerable<T> and uses yield return to lazily produce values. The caller picks the traversal strategy they need:

//       4
//      / 
//     2   6
//    /  / 
//   1  3 5  7
var tree = new BinaryTreeNode<int>(4,
    new BinaryTreeNode<int>(2,
        new BinaryTreeNode<int>(1),
        new BinaryTreeNode<int>(3)),
    new BinaryTreeNode<int>(6,
        new BinaryTreeNode<int>(5),
        new BinaryTreeNode<int>(7)));

// In-order: 1, 2, 3, 4, 5, 6, 7
foreach (var value in tree.InOrder())
{
    Console.Write($"{value} ");
}

Console.WriteLine();

// Pre-order: 4, 2, 1, 3, 6, 5, 7
foreach (var value in tree.PreOrder())
{
    Console.Write($"{value} ");
}

This is where the composite design pattern and the iterator pattern complement each other. Composite gives you a uniform interface for treating individual objects and compositions the same way. The iterator pattern gives you a uniform way to traverse that composition. When your composite structure represents a file system, an organization chart, or a UI component tree, exposing IEnumerable<T> traversals lets consumers work with the hierarchy without understanding its shape.

Scenario: Paginated API Results

APIs that return paginated results are a strong case for when to use the iterator pattern in C#. The consumer wants to iterate over all results. The iterator handles fetching page by page behind the scenes. Client code sees a simple foreach and never thinks about page tokens, offsets, or batch sizes.

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Text.Json;

public sealed class PaginatedApiIterator<T>
{
    private readonly HttpClient _httpClient;
    private readonly string _baseUrl;
    private readonly int _pageSize;

    public PaginatedApiIterator(
        HttpClient httpClient,
        string baseUrl,
        int pageSize = 50)
    {
        _httpClient = httpClient;
        _baseUrl = baseUrl;
        _pageSize = pageSize;
    }

    public IEnumerable<T> GetAll()
    {
        int page = 1;
        bool hasMore = true;

        while (hasMore)
        {
            var url = $"{_baseUrl}?page={page}" +
                      $"&pageSize={_pageSize}";

            var response = _httpClient
                .GetStringAsync(url)
                .GetAwaiter()
                .GetResult();

            var pageResult = JsonSerializer
                .Deserialize<PageResult<T>>(response);

            if (pageResult?.Items is null ||
                pageResult.Items.Count == 0)
            {
                yield break;
            }

            foreach (var item in pageResult.Items)
            {
                yield return item;
            }

            hasMore = pageResult.Items.Count == _pageSize;
            page++;
        }
    }
}

public sealed class PageResult<T>
{
    public List<T>? Items { get; set; }

    public int TotalCount { get; set; }
}

The consumer code is clean and completely unaware of pagination:

var httpClient = new HttpClient();
var iterator = new PaginatedApiIterator<UserDto>(
    httpClient,
    "https://api.example.com/users",
    pageSize: 25);

foreach (var user in iterator.GetAll())
{
    Console.WriteLine($"{user.Name} ({user.Email})");
}

There's an important note here. This synchronous example uses GetAwaiter().GetResult() for simplicity. In production, you'd use an async version with IAsyncEnumerable<T> and await foreach instead. The pattern is the same -- the iterator abstracts away the page-fetching mechanics -- but the async variant avoids blocking threads.

For a production-quality async version:

public async IAsyncEnumerable<T> GetAllAsync()
{
    int page = 1;
    bool hasMore = true;

    while (hasMore)
    {
        var url = $"{_baseUrl}?page={page}" +
                  $"&pageSize={_pageSize}";

        var response = await _httpClient
            .GetStringAsync(url);

        var pageResult = JsonSerializer
            .Deserialize<PageResult<T>>(response);

        if (pageResult?.Items is null ||
            pageResult.Items.Count == 0)
        {
            yield break;
        }

        foreach (var item in pageResult.Items)
        {
            yield return item;
        }

        hasMore = pageResult.Items.Count == _pageSize;
        page++;
    }
}

This pattern works for any paginated source -- REST APIs, database cursors, or even file-based batch processing. The key benefit is encapsulation: the consumer iterates without knowing about the pagination protocol. If the API changes its pagination scheme from page numbers to cursor tokens, you update one method and every consumer continues working unchanged. This kind of encapsulation is a core benefit you'll see across many design patterns, including the chain of responsibility pattern where each handler hides processing details from the caller.

When NOT to Use the Iterator Pattern

Knowing when to use the iterator pattern in C# means also knowing when it's the wrong tool. The pattern adds value when you need lazy evaluation, custom traversal, or data source abstraction. In other cases, it introduces complexity without a payoff.

Simple Lists and Arrays

If your data is already in a List<T>, an array, or any other built-in collection, there's no reason to use the iterator pattern in C# for a custom implementation. The foreach keyword works out of the box. These types implement IEnumerable<T> and provide efficient enumerators. Writing a custom iterator around an existing collection doesn't add value -- it adds a layer of indirection that makes debugging harder and code longer.

Random Access Requirements

Iterators are sequential by design. You move forward through a sequence, one element at a time. If your consumer needs to jump to element 47, access the last element, or index by position, an iterator is the wrong abstraction. Use an indexer, a List<T>, or an array where collection[index] gives you O(1) access.

All Elements Must Be Available at Once

Some algorithms need the full dataset in memory before they can proceed -- sorting, grouping, aggregating, or finding the median all require the complete collection. If your consumers consistently call .ToList() or .ToArray() on your IEnumerable<T> before doing any work, using the iterator pattern in C# adds overhead without benefit. You're paying the cost of the iterator state machine and then materializing everything anyway. Return a List<T> or IReadOnlyList<T> instead and make the intent clear.

There's one more situation worth flagging.

Deferred Execution Causing Subtle Bugs

Deferred execution means your iterator's body doesn't run until someone enumerates it. This is usually a feature, but it can cause problems when the data source changes between when you create the iterator and when you consume it. If you yield from a database connection and the connection gets disposed before enumeration starts, you'll get an exception at enumeration time -- not at creation time. This makes bugs harder to trace. When the lifetime of your data source doesn't align with enumeration timing, the iterator pattern in C# can work against you. Consider materializing the results eagerly or using a pattern like the flyweight pattern to share computed results efficiently.

Iterator Pattern in the .NET Ecosystem

The iterator pattern isn't just a textbook concept in C# -- it's woven into the framework at every level. Understanding the iterator pattern in C# makes you better at using the tools .NET provides.

LINQ is built on iterators. Every LINQ method like Where, Select, and Take returns an IEnumerable<T> that uses deferred execution. When you chain .Where(x => x > 5).Select(x => x * 2).Take(10), no work happens until you enumerate. Each operator is an iterator that pulls from the previous one. Understanding the iterator pattern explains why LINQ is lazy, why calling .ToList() triggers execution, and why you can compose queries without performance penalties until enumeration begins.

IAsyncEnumerable<T> extends the pattern to async. When you need to iterate over data that arrives asynchronously -- reading from a gRPC stream, consuming messages from a channel, or querying a database with streaming results -- IAsyncEnumerable<T> is the async version of the iterator pattern. The await foreach syntax works identically to foreach, and yield return inside an async method produces the async iterator state machine. If you've worked with dependency injection through IServiceCollection, you've seen how .NET wires up services behind a clean API. IAsyncEnumerable<T> does the same for async data streams -- it hides the complexity of async iteration behind a simple loop.

Channels use the pattern for producer-consumer scenarios. System.Threading.Channels provides bounded and unbounded channels where producers write and consumers read. The ChannelReader<T>.ReadAllAsync() method returns an IAsyncEnumerable<T>, letting you consume channel items with await foreach. Without understanding the iterator pattern, the relationship between channels and async enumeration isn't obvious.

System.IO.Pipelines leverages sequential reading. While pipelines use a different API than IEnumerable<T>, the conceptual model is the same: you read data incrementally from a source without loading everything into memory. The iterator pattern gives you the mental model to understand why pipelines work the way they do and when to choose them over simpler IEnumerable<T> based approaches.

The takeaway is straightforward. Understanding when to use the iterator pattern in C# isn't just about learning an academic design pattern -- you're learning the foundation that powers LINQ, async streams, channels, and half the APIs in the .NET Base Class Library.

Frequently Asked Questions

What is the difference between IEnumerable and IEnumerator in C#?

IEnumerable<T> represents a sequence that can be iterated. It has one method: GetEnumerator(), which returns an IEnumerator<T>. The enumerator is the actual cursor that tracks where you are in the sequence. It has Current (the element you're pointing at) and MoveNext() (advance to the next element). Think of IEnumerable<T> as the collection and IEnumerator<T> as the bookmark. When you use yield return in C#, the compiler generates both for you automatically, which is why you rarely implement IEnumerator<T> by hand.

Should I use yield return or implement IEnumerator manually?

Use yield return in the vast majority of cases. The C# compiler generates the state machine code that IEnumerator<T> requires, handling MoveNext, Current, Reset, and Dispose for you. Manual implementation is only justified when you need fine-grained control over the state machine -- for example, when building a high-performance enumerator that avoids heap allocations by using a struct enumerator. For everyday scenarios like file processing, tree traversal, and API pagination, yield return produces correct, readable, and maintainable code.

How does yield return work under the hood in C#?

When the C# compiler encounters yield return, it transforms your method into a state machine class that implements IEnumerable<T> and IEnumerator<T>. Each yield return becomes a state transition. Local variables become fields on the generated class so their values persist across calls to MoveNext(). When the consumer calls MoveNext(), execution resumes from where the last yield return suspended. This is why iterator methods don't execute when you call them -- they return the state machine object. Execution only begins when the consumer starts enumerating.

Can I use the iterator pattern with async code in C#?

Yes. C# supports async iterator methods that return IAsyncEnumerable<T>. You can use yield return inside an async method, and consumers iterate with await foreach. This is essential for streaming data from async sources -- databases, HTTP APIs, message queues, and gRPC streams all benefit from async iteration. The compiler generates an async state machine that handles both the await suspension points and the yield return suspension points, giving you the same lazy evaluation semantics you get with synchronous iterators.

When should I call ToList() instead of keeping an IEnumerable?

Call ToList() when you need to iterate the sequence more than once, when you need Count or index access without re-enumerating, or when the underlying data source might change or be disposed before you finish enumerating. Keep the IEnumerable<T> when you want lazy evaluation, when you might not need all elements, or when memory is a concern and you're working with large datasets. A common mistake is calling ToList() on a LINQ query deep inside a method and then returning IEnumerable<T> from the method -- you've already paid the materialization cost, so return IReadOnlyList<T> to communicate the intent clearly.

How does the iterator pattern relate to the observer pattern?

The iterator pattern and the observer pattern are duals of each other. The iterator pattern is "pull-based" -- the consumer requests the next element when it's ready. The observer pattern is "push-based" -- the producer sends elements to subscribers when they're available. In .NET, IEnumerable<T> represents the pull model and IObservable<T> represents the push model. Understanding both helps you choose the right approach for your data flow. If the consumer controls the pace, use iterators. If the producer controls the pace, use observers.

Is the iterator pattern thread-safe in C#?

Not by default. The enumerator state machine generated by yield return is not thread-safe. If multiple threads enumerate the same IEnumerable<T> instance concurrently, each gets its own enumerator (from GetEnumerator()), so that's fine. But if multiple threads share a single IEnumerator<T> and call MoveNext() concurrently, the behavior is undefined. For concurrent scenarios, use IAsyncEnumerable<T> with proper async coordination, or materialize to a thread-safe collection first. Channels in .NET provide a thread-safe producer-consumer model that integrates with IAsyncEnumerable<T> for safe concurrent iteration.