ILangfuseExperimentRun

NexusLabs.Foundry.Langfuse

ILangfuseExperimentRun Interface

Represents an in-progress Langfuse experiment (dataset run). Each call to RunItemAsync<T>(string, Func<ILangfuseScenario,CancellationToken,Task<T>>, LangfuseExperimentItemOptions, CancellationToken) executes one dataset item inside an active scenario trace and links it to this run, so scores recorded on that trace roll up into the experiment-comparison view.

public interface ILangfuseExperimentRun

Remarks

The referenced dataset and its items must already exist (see ILangfuseDatasetClient). The run itself is created implicitly by Langfuse on the first linked item. Run names are caller-supplied (for example a git SHA or CI run id) so runs are comparable and reproducible.

Properties

ILangfuseExperimentRun.DatasetName Property

Gets the dataset this run is scored against.

string DatasetName { get; }

Property Value

System.String

ILangfuseExperimentRun.DatasetRunId Property

Gets the authoritative dataset-run id after successful links agree on one identity.

string? DatasetRunId { get; }

Property Value

System.String

ILangfuseExperimentRun.DatasetVersion Property

Gets the optional hosted dataset version timestamp used for every item link.

System.Nullable<System.DateTimeOffset> DatasetVersion { get; }

Property Value

System.Nullable<System.DateTimeOffset>

ILangfuseExperimentRun.Description Property

Gets the requested run description.

string? Description { get; }

Property Value

System.String

ILangfuseExperimentRun.IdentityStatus Property

Gets the aggregate dataset-run identity status.

NexusLabs.Foundry.Langfuse.LangfuseDatasetRunIdentityStatus IdentityStatus { get; }

Property Value

LangfuseDatasetRunIdentityStatus

ILangfuseExperimentRun.Metadata Property

Gets the frozen structured metadata submitted with item links.

System.Nullable<System.Text.Json.JsonElement> Metadata { get; }

Property Value

System.Nullable<System.Text.Json.JsonElement>

ILangfuseExperimentRun.RunName Property

Gets the run name (for example a git SHA or CI run id).

string RunName { get; }

Property Value

System.String

Methods

ILangfuseExperimentRun.GetPublicationSnapshot() Method

Gets an immutable snapshot of direct item-link and run-score API outcomes observed by this run instance.

NexusLabs.Foundry.Langfuse.LangfuseExperimentRunPublicationSnapshot GetPublicationSnapshot();

Returns

LangfuseExperimentRunPublicationSnapshot
The current publication snapshot.

ILangfuseExperimentRun.RecordEvaluationAsync(EvaluationResult) Method

Projects every metric in result to a score against the resolved dataset run.

System.Threading.Tasks.Task<System.Collections.Generic.IReadOnlyList<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult>> RecordEvaluationAsync(Microsoft.Extensions.AI.Evaluation.EvaluationResult result);

Parameters

result Microsoft.Extensions.AI.Evaluation.EvaluationResult

The evaluation result to project.

Returns

System.Threading.Tasks.Task<System.Collections.Generic.IReadOnlyList<LangfuseExperimentRunScoreResult>>
One structured score result per evaluation metric.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

ILangfuseExperimentRun.RecordEvaluationAsync(EvaluationResult, LangfuseEvaluationScoreOptions, CancellationToken) Method

Projects every metric in result to a score against the resolved dataset run.

System.Threading.Tasks.Task<System.Collections.Generic.IReadOnlyList<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult>> RecordEvaluationAsync(Microsoft.Extensions.AI.Evaluation.EvaluationResult result, NexusLabs.Foundry.Langfuse.LangfuseEvaluationScoreOptions? options, System.Threading.CancellationToken cancellationToken);

Parameters

result Microsoft.Extensions.AI.Evaluation.EvaluationResult

The evaluation result to project.

options LangfuseEvaluationScoreOptions

Optional stable identity settings for projected metric scores.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<System.Collections.Generic.IReadOnlyList<LangfuseExperimentRunScoreResult>>
One structured score result per evaluation metric.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

System.OperationCanceledException
cancellationToken was canceled.

ILangfuseExperimentRun.RecordScoreAsync(string, bool) Method

Records a boolean score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, bool value);

Parameters

name System.String

The score name.

value System.Boolean

The boolean value.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

ILangfuseExperimentRun.RecordScoreAsync(string, bool, LangfuseScoreOptions, CancellationToken) Method

Records a boolean score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, bool value, NexusLabs.Foundry.Langfuse.LangfuseScoreOptions? options, System.Threading.CancellationToken cancellationToken);

Parameters

name System.String

The score name.

value System.Boolean

The boolean value.

options LangfuseScoreOptions

Optional score identity and comment settings.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

System.OperationCanceledException
cancellationToken was canceled.

ILangfuseExperimentRun.RecordScoreAsync(string, double) Method

Records a numeric score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, double value);

Parameters

name System.String

The score name.

value System.Double

The numeric value.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

ILangfuseExperimentRun.RecordScoreAsync(string, double, LangfuseScoreOptions, CancellationToken) Method

Records a numeric score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, double value, NexusLabs.Foundry.Langfuse.LangfuseScoreOptions? options, System.Threading.CancellationToken cancellationToken);

Parameters

name System.String

The score name.

value System.Double

The numeric value.

options LangfuseScoreOptions

Optional score identity and comment settings.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

System.OperationCanceledException
cancellationToken was canceled.

ILangfuseExperimentRun.RecordScoreAsync(string, string) Method

Records a categorical score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, string value);

Parameters

name System.String

The score name.

value System.String

The category label.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

ILangfuseExperimentRun.RecordScoreAsync(string, string, LangfuseScoreOptions, CancellationToken) Method

Records a categorical score against the resolved dataset run.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentRunScoreResult> RecordScoreAsync(string name, string value, NexusLabs.Foundry.Langfuse.LangfuseScoreOptions? options, System.Threading.CancellationToken cancellationToken);

Parameters

name System.String

The score name.

value System.String

The category label.

options LangfuseScoreOptions

Optional score identity and comment settings.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<LangfuseExperimentRunScoreResult>
The direct score-publication outcome.

Exceptions

LangfuseException
The dataset-run identity is unavailable or score publication failed while strict score mode is configured.

System.OperationCanceledException
cancellationToken was canceled.

ILangfuseExperimentRun.RunItemAsync<T>(string, Func<ILangfuseScenario,CancellationToken,Task<T>>) Method

Executes a callback while one dataset item's scenario is active and links the scenario trace to this run as a dataset-run-item.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentItemResult<T>> RunItemAsync<T>(string datasetItemId, System.Func<NexusLabs.Foundry.Langfuse.ILangfuseScenario,System.Threading.CancellationToken,System.Threading.Tasks.Task<T>> callback);

Type parameters

T

The callback result type.

Parameters

datasetItemId System.String

The id of the dataset item being evaluated.

callback System.Func<ILangfuseScenario,System.Threading.CancellationToken,System.Threading.Tasks.Task<T>>

The item work to execute. The supplied scenario is active as System.Diagnostics.Activity.Current for the callback lifetime and is disposed before this method completes.

Returns

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentItemResult<T>>
The callback value, trace id, and structured dataset-link result.

Exceptions

System.ArgumentException
datasetItemId is null or whitespace.

System.ArgumentNullException
callback is null.

LangfuseException
Langfuse could not link the item and strict link-failure mode was selected.

Remarks

The scenario is valid only for the callback lifetime and must not be retained. Callback exceptions and caller-requested cancellation propagate unchanged after the scenario is disposed and the previous ambient activity is restored.

ILangfuseExperimentRun.RunItemAsync<T>(string, Func<ILangfuseScenario,CancellationToken,Task<T>>, LangfuseExperimentItemOptions, CancellationToken) Method

Executes a callback while one dataset item's scenario is active and links the scenario trace to this run as a dataset-run-item.

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentItemResult<T>> RunItemAsync<T>(string datasetItemId, System.Func<NexusLabs.Foundry.Langfuse.ILangfuseScenario,System.Threading.CancellationToken,System.Threading.Tasks.Task<T>> callback, NexusLabs.Foundry.Langfuse.LangfuseExperimentItemOptions? options, System.Threading.CancellationToken cancellationToken);

Type parameters

T

The callback result type.

Parameters

datasetItemId System.String

The id of the dataset item being evaluated.

callback System.Func<ILangfuseScenario,System.Threading.CancellationToken,System.Threading.Tasks.Task<T>>

The item work to execute. The supplied scenario is active as System.Diagnostics.Activity.Current for the callback lifetime and is disposed before this method completes.

options LangfuseExperimentItemOptions

Optional scenario and dataset-link behavior.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<NexusLabs.Foundry.Langfuse.LangfuseExperimentItemResult<T>>
The callback value, trace id, and structured dataset-link result.

Exceptions

System.ArgumentException
datasetItemId is null or whitespace.

System.ArgumentNullException
callback is null.

System.ArgumentOutOfRangeException
LinkFailureMode is not defined.

LangfuseException
Langfuse could not link the item and strict link-failure mode was selected.

System.OperationCanceledException
cancellationToken was canceled before or during linking or callback execution.

Remarks

The scenario is valid only for the callback lifetime and must not be retained. Callback exceptions and caller-requested cancellation propagate unchanged after the scenario is disposed and the previous ambient activity is restored.