EvaluationQualityGate

NexusLabs.Foundry.Evaluation

NexusLabs.Foundry.Evaluation

EvaluationQualityGate Class

Configurable quality gate that asserts evaluation metrics meet defined thresholds. Designed for CI pipelines — call Assert(EvaluationResult[]) after running evaluators to fail the build when metrics regress.

public sealed class EvaluationQualityGate

Inheritance System.Object 🡒 EvaluationQualityGate

Example

var gate = new EvaluationQualityGate()
    .RequireBoolean(ToolCallTrajectoryEvaluator.AllSucceededMetricName, expected: true)
    .RequireBoolean(IterationCoherenceEvaluator.TerminatedCoherentlyMetricName, expected: true)
    .RequireNumericMax(EfficiencyEvaluator.TotalTokensMetricName, max: 50_000)
    .RequireBoolean(EfficiencyEvaluator.UnderBudgetMetricName, expected: true);

// Throws QualityGateFailedException if any threshold is violated.
gate.Assert(trajectoryResult, coherenceResult, efficiencyResult);

Remarks

Thresholds are defined fluently via RequireNumericMax(string, double), RequireNumericMin(string, double), and RequireBoolean(string, bool). Each threshold names a metric (using the *MetricName constants from evaluator classes) and a bound. Assert(EvaluationResult[]) checks all thresholds against the evaluation result and throws QualityGateFailedException when the structured decision is failed or inconclusive.

Required missing or invalid metrics produce an inconclusive decision. Use the Optional* methods for metrics that are conditionally emitted.

Methods

EvaluationQualityGate.Assert(EvaluationResult[]) Method

Checks all thresholds against the provided evaluation results. Metrics are looked up across all results — the first match wins.

public void Assert(params Microsoft.Extensions.AI.Evaluation.EvaluationResult[] results);

Parameters

results Microsoft.Extensions.AI.Evaluation.EvaluationResult[]

One or more Microsoft.Extensions.AI.Evaluation.EvaluationResult instances to check.

Exceptions

QualityGateFailedException
Thrown when the structured threshold decision is failed or inconclusive. The exception message lists every failed, missing, or invalid required threshold.

EvaluationQualityGate.Evaluate(EvaluationResult[]) Method

Evaluates all configured thresholds without throwing.

public NexusLabs.Foundry.Evaluation.EvaluationThresholdResult Evaluate(params Microsoft.Extensions.AI.Evaluation.EvaluationResult[] results);

Parameters

results Microsoft.Extensions.AI.Evaluation.EvaluationResult[]

One or more evaluation results to check.

Returns

EvaluationThresholdResult
The structured threshold result.

EvaluationQualityGate.Evaluate(EvaluationMissingMetricBehavior, EvaluationResult[]) Method

Evaluates all configured thresholds without throwing.

public NexusLabs.Foundry.Evaluation.EvaluationThresholdResult Evaluate(NexusLabs.Foundry.Evaluation.EvaluationMissingMetricBehavior missingMetricBehavior, params Microsoft.Extensions.AI.Evaluation.EvaluationResult[] results);

Parameters

missingMetricBehavior EvaluationMissingMetricBehavior

The treatment for required missing or invalid metrics.

results Microsoft.Extensions.AI.Evaluation.EvaluationResult[]

One or more evaluation results to check.

Returns

EvaluationThresholdResult
The structured threshold result.

EvaluationQualityGate.OptionalBoolean(string, bool) Method

Checks a Microsoft.Extensions.AI.Evaluation.BooleanMetric value when the metric is present.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate OptionalBoolean(string metricName, bool expected);

Parameters

metricName System.String

The metric name.

expected System.Boolean

The required value when present.

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.OptionalNumericMax(string, double) Method

Checks a Microsoft.Extensions.AI.Evaluation.NumericMetric maximum when the metric is present.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate OptionalNumericMax(string metricName, double max);

Parameters

metricName System.String

The metric name.

max System.Double

The maximum allowed value.

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.OptionalNumericMin(string, double) Method

Checks a Microsoft.Extensions.AI.Evaluation.NumericMetric minimum when the metric is present.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate OptionalNumericMin(string metricName, double min);

Parameters

metricName System.String

The metric name.

min System.Double

The minimum allowed value.

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.RequireBoolean(string, bool) Method

Requires a Microsoft.Extensions.AI.Evaluation.BooleanMetric to equal expected.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate RequireBoolean(string metricName, bool expected);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

expected System.Boolean

The required boolean value.

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.RequireNumericMax(string, double) Method

Requires a Microsoft.Extensions.AI.Evaluation.NumericMetric to be at most max.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate RequireNumericMax(string metricName, double max);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

max System.Double

The maximum allowed value (inclusive).

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.RequireNumericMin(string, double) Method

Requires a Microsoft.Extensions.AI.Evaluation.NumericMetric to be at least min.

public NexusLabs.Foundry.Evaluation.EvaluationQualityGate RequireNumericMin(string metricName, double min);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

min System.Double

The minimum allowed value (inclusive).

Returns

EvaluationQualityGate
This gate instance for fluent chaining.