RAG Query Transformation in .NET: HyDE, Step-Back, and Decomposition

RAG query transformation in .NET is a retrieval-augmented generation (RAG) technique that changes the retrieval input before a search system sees it. The user still asked one question. The application may also search with a hypothetical answer, a broader principle, or several narrower questions. That distinction matters: transformation changes how evidence is found, not what the final answer is allowed to claim.

The baseline remains valuable. Embed the original question, retrieve candidates, and inspect the results. A transform is an experiment against that baseline, not an automatic upgrade. For a framework-specific end-to-end starting point, see my RAG with Semantic Kernel guide. This article stays at the pre-retrieval boundary, using small .NET contracts that do not assume a particular model client or vector store.

Start RAG query transformation in .NET with the unmodified question

Before adding another model call, define what the original query retrieves. The baseline establishes the vocabulary supplied by the user, the embedding model, retrieval filters, candidate count, and corpus revision. Without it, a transformed result that looks plausible has nothing meaningful to beat.

In a typical dense-retrieval flow, the question becomes a vector and a retriever returns scored chunks. Microsoft documents IEmbeddingGenerator<TInput, TEmbedding>, Embedding<float>.Vector, and the single-input GenerateVectorAsync helper. That gives RAG query transformation in .NET a stable seam after a transformed string exists. It does not provide HyDE, step-back prompting, or decomposition as built-in APIs.

The examples use Microsoft.Extensions.AI.Abstractions 10.7.0 and target .NET 8; they were validated against the current GenerateVectorAsync API reference, whose optional EmbeddingGenerationOptions parameter precedes CancellationToken.

using Microsoft.Extensions.AI;

public sealed record RetrievalHit(
    string DocumentId,
    string ChunkId,
    string Text,
    double Score);

public interface IVectorRetriever
{
    Task<IReadOnlyList<RetrievalHit>> SearchAsync(
        ReadOnlyMemory<float> queryVector,
        CancellationToken cancellationToken);
}

public sealed class BaselineQueryRetriever(
    IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator,
    IVectorRetriever retriever)
{
    public async Task<IReadOnlyList<RetrievalHit>> SearchAsync(
        string question,
        CancellationToken cancellationToken)
    {
        ReadOnlyMemory<float> vector = await embeddingGenerator
            .GenerateVectorAsync(question, cancellationToken: cancellationToken);

        return await retriever.SearchAsync(vector, cancellationToken);
    }
}

The code intentionally has no HydeAsync, StepBackAsync, or DecomposeAsync method. Those names would imply a library contract that Microsoft.Extensions.AI does not define. A text-producing component belongs on the other side of a local interface, while embedding and retrieval remain independently testable.

This also keeps a helpful architectural boundary with the existing document Q&A application. Query transformation is not another document ingestion flow, an agent loop, or a web-search fallback. It is a decision about the search inputs for an already available corpus.

Treat transformed text as additional retrieval keys

The simplest useful contract produces zero or more strings from the original question. A baseline-only policy returns the original question. A transformation policy may return the original question plus one or more additional keys. The retrieval layer does not need to know why a particular key exists.

using Microsoft.Extensions.AI;

public delegate Task<IReadOnlyList<string>> QueryTransform(
    string question,
    CancellationToken cancellationToken);

public sealed class TransformedQueryRetriever(
    IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator,
    IVectorRetriever retriever,
    QueryTransform transform)
{
    public async Task<IReadOnlyList<IReadOnlyList<RetrievalHit>>> SearchAsync(
        string question,
        CancellationToken cancellationToken)
    {
        IReadOnlyList<string> retrievalKeys = await transform(
            question,
            cancellationToken);

        var searches = retrievalKeys.Select(async key =>
        {
            ReadOnlyMemory<float> vector = await embeddingGenerator
                .GenerateVectorAsync(key, cancellationToken: cancellationToken);

            return await retriever.SearchAsync(vector, cancellationToken);
        });

        return await Task.WhenAll(searches);
    }
}

QueryTransform is deliberately provider-neutral. Its implementation might call a model, apply a deterministic rewrite rule, or return a reviewed lookup expansion. The application owns validation, logging policy, cancellation, and the decision to send generated text to retrieval. The model-specific instructions that produce text are not a portable .NET API, so they should stay in the provider adapter rather than masquerading as one.

The original question should usually remain among the keys. It preserves a direct route to exact terminology, identifiers, and wording that a more abstract form can lose. The candidate sets then need an explicit merge policy, which we will return to after looking at the three research-backed transformations.

HyDE embeds a hypothetical document, not an answer

HyDE, short for Hypothetical Document Embeddings, was introduced for zero-shot dense retrieval. The method asks a language model to generate a hypothetical document relevant to the question, embeds that generated document, and uses its vector to retrieve real documents from the corpus. The original paper explicitly notes that the generated document can contain incorrect details. It is a retrieval key, not evidence and not text to show to the reader.

That is the important mental model for RAG query transformation in .NET. A terse question such as “Why does an index return a stale policy?” may share more embedding-space vocabulary with an explanatory passage than with the exact words in the corpus. A hypothetical explanatory document can bridge that vocabulary gap. Retrieval should still return corpus chunks, and final grounding must come from those chunks.

The implementation boundary should make that restriction visible: the hypothetical document is an additional key, while the original question is retained. The local QueryTransform seam above is enough to express that rule without turning the text-generation instruction into an invented API. A downstream answer composer should receive retrieved chunks and their provenance, never the hypothetical document as a citation candidate.

HyDE is not a substitute for measuring retrieval quality on the corpus at hand. The paper reports research results for its zero-shot setting, but that does not establish a universal improvement for a different embedding model, domain, or query distribution. A corpus dominated by exact product names, error codes, or short identifiers may behave quite differently from one with explanatory prose.

Step-back transformation searches for the governing idea

Step-Back Prompting describes deriving high-level concepts and first principles from a detail-heavy question, then using that abstraction to guide reasoning. At retrieval time, the useful adaptation is a second search key that expresses the governing concept behind the specific request.

For example, a question about a precise failure may benefit from a broader key about cache invalidation, authorization filtering, versioning, or consistency. The transform does not answer the original question. It names the concept under which the corpus may explain the question.

This is distinct from paraphrasing. A paraphrase preserves approximately the same specificity. A step-back key intentionally moves upward:

The direction is useful only if the corpus contains conceptual material that helps answer the specific question. A step-back key can also become too broad and retrieve introductory material that displaces evidence about the real subject. Keep it as one identifiable retrieval key, merge it with the baseline, and inspect the changed candidate set.

There is a natural connection here to how content was prepared. Chunk boundaries determine whether a broad principle and a specific procedure coexist in one chunk or live in different parts of a source. If you are debugging that relationship, my chunking strategies article covers the framework-specific mechanics. Transformation cannot repair a corpus whose relevant facts were never indexed in retrievable form.

Decomposition makes multi-part evidence visible

Question decomposition addresses a different retrieval failure. Some questions require facts from several sources. Searching the whole question as one vector can underrepresent the individual facts needed to resolve it.

The Question Decomposition for Retrieval-Augmented Generation paper describes a pipeline that decomposes a multi-hop question into sub-questions, retrieves passages for each, merges candidates, and reranks the pool. Its premise is useful even when your own implementation uses a different retriever: each sub-question should be independently understandable enough to retrieve evidence for one part of the overall answer.

Do not decompose every query by default. A direct factual question may only create duplicate searches. Decomposition becomes a candidate when a question has independently retrievable parts, such as a comparison requiring two facts, a sequence requiring a prerequisite, or a question whose conclusion depends on evidence distributed across documents.

The result needs structure so the application can preserve which sub-question produced which candidates:

public sealed record SubQuestion(string Id, string Text);

public sealed record CandidateSet(
    string RetrievalKeyId,
    IReadOnlyList<RetrievalHit> Hits);

public static class Decomposition
{
    public static IReadOnlyList<SubQuestion> KeepDistinct(
        IEnumerable<SubQuestion> subQuestions)
    {
        return subQuestions
            .Where(question => !string.IsNullOrWhiteSpace(question.Text))
            .DistinctBy(question => question.Text.Trim(),
                StringComparer.OrdinalIgnoreCase)
            .ToArray();
    }
}

The transform producer supplies the sub-questions; this BCL-only code keeps stable IDs and removes repeated text. That provenance is practical. When a merged result is surprising, you can tell whether it came from the baseline, an abstraction, a hypothetical document, or one particular decomposed question. It also prevents “many searches happened” from becoming the only explanation available in a trace.

Merge candidates into an identity-qualified pool

Multiple retrieval keys create multiple candidate lists. The question-decomposition paper aggregates passages found for sub-questions, then applies a reranker to the expanded pool against the original complex question. This sample performs only the aggregation part.

Each candidate has a composite (DocumentId, ChunkId) identity, so a chunk ID that is only unique within one document cannot merge with a chunk from another document. The result is a deterministic, identity-ordered pool that retains the retrieval-key provenance and deliberately performs no raw-score combination or relevance ordering. A separately evaluated fusion or reranking policy can order the pool later.

public sealed record CandidateIdentity(
    string DocumentId,
    string ChunkId);

public sealed record MergedCandidate(
    CandidateIdentity Identity,
    IReadOnlySet<string> RetrievalKeyIds);

public static class CandidateMerge
{
    public static IReadOnlyList<MergedCandidate> ByDocumentAndChunk(
        IEnumerable<CandidateSet> candidateSets)
    {
        return candidateSets
            .SelectMany(set => set.Hits.Select(hit => (set.RetrievalKeyId, hit)))
            .GroupBy(item => new CandidateIdentity(
                item.hit.DocumentId,
                item.hit.ChunkId))
            .OrderBy(group => group.Key.DocumentId, StringComparer.Ordinal)
            .ThenBy(group => group.Key.ChunkId, StringComparer.Ordinal)
            .Select(group => new MergedCandidate(
                group.Key,
                group
                    .Select(item => item.RetrievalKeyId)
                    .ToHashSet(StringComparer.Ordinal)))
            .ToArray();
    }
}

Candidate merging is not reranking. The merge decides which identity-qualified results are available for a later selection step; a ranking policy decides their preferred order. The cited decomposition pipeline makes the same aggregation-then-reranking distinction, so its evaluation does not imply that this sample's unranked pool is already a final result list.

If you need a concrete semantic-search implementation, the existing Semantic Kernel semantic search guide is a useful companion. Its framework-specific implementation is not a reason to couple your transformation policy to a Semantic Kernel type.

Evaluate RAG query transformation in .NET against retrieval evidence

Every transform is a retrieval hypothesis that needs a baseline comparison. The question-decomposition paper reports retrieval and final-answer results separately, which is why retrieval evidence remains part of RAG query transformation in .NET, not a follow-up task.

For a local comparison, define reviewed questions with the document-qualified chunk IDs expected to support an answer. Run baseline retrieval and each transformation against the same corpus revision, embedding model, filters, and candidate budget, then inspect per-query outcomes before choosing an aggregate measurement.

Here is a deterministic recall-at-K helper. It measures whether expected document-qualified chunk IDs occur in the retrieved prefix; it does not judge whether a generated answer is correct.

public sealed record RetrievalJudgment(
    string QuestionId,
    IReadOnlySet<CandidateIdentity> ExpectedCandidateIds);

public static class RetrievalMetrics
{
    public static double RecallAtK(
        RetrievalJudgment judgment,
        IReadOnlyList<RetrievalHit> hits,
        int k)
    {
        if (judgment.ExpectedCandidateIds.Count == 0)
        {
            return 0;
        }

        int found = hits
            .Take(k)
            .Select(hit => new CandidateIdentity(
                hit.DocumentId,
                hit.ChunkId))
            .Distinct()
            .Count(judgment.ExpectedCandidateIds.Contains);

        return (double)found / judgment.ExpectedCandidateIds.Count;
    }
}

Record the transform type, retrieval keys, corpus revision, embedding model identifier, filters, top-K, merged candidate IDs, and judgment result for each comparison. Those fields let a reviewer inspect whether a changed result came from the transform, the retrieved pool, or the later ranking policy.

The original papers motivate these methods, but they do not provide a transferable result for your corpus. Treat the baseline as a control, and keep a reviewed case set that includes direct questions, vocabulary gaps, abstraction-heavy questions, and multi-hop questions. That is a better foundation than a universal claim about HyDE, step-back, or decomposition.

Frequently asked questions

These questions separate the retrieval role of each transform from the evidence and evaluation responsibilities that remain unchanged.

What is RAG query transformation in .NET?

RAG query transformation in .NET creates one or more retrieval keys from a user question before embedding and search. The generated strings change candidate discovery; retrieved corpus content remains the evidence for an answer.

Should I always use HyDE for dense retrieval?

No. HyDE is a research-backed way to create a hypothetical-document retrieval key, but its usefulness depends on the corpus, embedding model, and query set. Measure it against the original-question baseline.

Is step-back prompting the same as rewriting a query?

No. A rewrite usually preserves the question's level of detail. A step-back transformation deliberately searches for a broader principle or concept that may explain the detailed question.

When should a RAG query be decomposed?

Consider decomposition when answering requires independently retrievable facts from multiple sources. Keep direct questions direct when breaking them apart would only repeat the same search.

Can I cite a HyDE-generated document in a RAG response?

No. A hypothetical document is a retrieval key and can contain invented details. Cite only the real corpus chunks that retrieval returned and the answer actually used.

How should I merge candidates from several transformed queries?

First deduplicate by a document-qualified chunk identity and retain the retrieval-key provenance. Then apply a separately evaluated fusion or reranking policy if needed; this sample intentionally leaves that policy out.

Keep the retrieval experiment honest

The durable design for RAG query transformation in .NET is modest: retain the original question, make each transformed key identifiable, embed every key through the same verified seam, merge candidates with provenance, and compare the result with a reviewed baseline. HyDE changes the retrieval representation. Step-back transformation changes the level of abstraction. Decomposition changes one question into several evidence requests.

Those are different hypotheses about how a corpus expresses knowledge. Treat them that way. The result is a retrieval system that can learn from its corpus instead of one that quietly accumulates clever prompts.