Data Models Reference

Complete reference for all data models, request/response classes, and enums.

Core Models

This section documents all data models used throughout the HoneyHive SDK.

Generated Models

All request and response models generated from the API schema.

class honeyhive.models.generated.SessionStartRequest(*, project, session_name, source, session_id=None, children_ids=None, config=None, inputs=None, outputs=None, error=None, duration=None, user_properties=None, metrics=None, feedback=None, metadata=None, start_time=None, end_time=None)[source]

Bases: BaseModel

Parameters:
project: str
session_name: str
source: str
session_id: str | None
children_ids: List[str] | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
duration: float | None
user_properties: Dict[str, Any] | None
metrics: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metadata: Dict[str, Any] | None
start_time: float | None
end_time: int | None
class honeyhive.models.generated.SessionPropertiesBatch(*, session_name=None, source=None, session_id=None, config=None, inputs=None, outputs=None, error=None, user_properties=None, metrics=None, feedback=None, metadata=None)[source]

Bases: BaseModel

Parameters:
session_name: str | None
source: str | None
session_id: str | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
user_properties: Dict[str, Any] | None
metrics: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.EventType(value)[source]

Bases: Enum

session = 'session'
model = 'model'
tool = 'tool'
chain = 'chain'
class honeyhive.models.generated.Event(*, project_id=None, source=None, event_name=None, event_type=None, event_id=None, session_id=None, parent_id=None, children_ids=None, config=None, inputs=None, outputs=None, error=None, start_time=None, end_time=None, duration=None, metadata=None, feedback=None, metrics=None, user_properties=None)[source]

Bases: BaseModel

Parameters:
project_id: str | None
source: str | None
event_name: str | None
event_type: EventType | None
event_id: str | None
session_id: str | None
parent_id: str | None
children_ids: List[str] | None
config: Dict[str, Any] | None
inputs: Dict[str, Any] | None
outputs: Dict[str, Any] | None
error: str | None
start_time: float | None
end_time: int | None
duration: float | None
metadata: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metrics: Dict[str, Any] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.generated.Operator(value)[source]

Bases: Enum

is_ = 'is'
is_not = 'is not'
contains = 'contains'
not_contains = 'not contains'
greater_than = 'greater than'
class honeyhive.models.generated.Type(value)[source]

Bases: Enum

string = 'string'
number = 'number'
boolean = 'boolean'
id = 'id'
class honeyhive.models.generated.EventFilter(*, field=None, value=None, operator=None, type=None)[source]

Bases: BaseModel

Parameters:
field: str | None
value: str | None
operator: Operator | None
type: Type | None
class honeyhive.models.generated.EventType1(value)[source]

Bases: Enum

model = 'model'
tool = 'tool'
chain = 'chain'
class honeyhive.models.generated.CreateEventRequest(*, project, source, event_name, event_type, event_id=None, session_id=None, parent_id=None, children_ids=None, config, inputs, outputs=None, error=None, start_time=None, end_time=None, duration, metadata=None, feedback=None, metrics=None, user_properties=None)[source]

Bases: BaseModel

Parameters:
project: str
source: str
event_name: str
event_type: EventType1
event_id: str | None
session_id: str | None
parent_id: str | None
children_ids: List[str] | None
config: Dict[str, Any]
inputs: Dict[str, Any]
outputs: Dict[str, Any] | None
error: str | None
start_time: float | None
end_time: int | None
duration: float
metadata: Dict[str, Any] | None
feedback: Dict[str, Any] | None
metrics: Dict[str, Any] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.generated.CreateModelEvent(*, project, model, provider, messages, response, duration, usage, cost=None, error=None, source=None, event_name=None, hyperparameters=None, template=None, template_inputs=None, tools=None, tool_choice=None, response_format=None)[source]

Bases: BaseModel

Parameters:
project: str
model: str
provider: str
messages: List[Dict[str, Any]]
response: Dict[str, Any]
duration: float
usage: Dict[str, Any]
cost: float | None
error: str | None
source: str | None
event_name: str | None
hyperparameters: Dict[str, Any] | None
template: List[Dict[str, Any]] | None
template_inputs: Dict[str, Any] | None
tools: List[Dict[str, Any]] | None
tool_choice: str | None
response_format: Dict[str, Any] | None
class honeyhive.models.generated.Type1(value)[source]

Bases: Enum

PYTHON = 'PYTHON'
LLM = 'LLM'
HUMAN = 'HUMAN'
COMPOSITE = 'COMPOSITE'
class honeyhive.models.generated.ReturnType(value)[source]

Bases: Enum

boolean = 'boolean'
float = 'float'
string = 'string'
categorical = 'categorical'
class honeyhive.models.generated.Threshold(*, min=None, max=None, pass_when=None, passing_categories=None)[source]

Bases: BaseModel

Parameters:
min: float | None
max: float | None
pass_when: bool | float | None
passing_categories: List[str] | None
class honeyhive.models.generated.Metric(*, name, type, criteria, description=None, return_type=None, enabled_in_prod=None, needs_ground_truth=None, sampling_percentage=None, model_provider=None, model_name=None, scale=None, threshold=None, categories=None, child_metrics=None, filters=None, id=None, created_at=None, updated_at=None)[source]

Bases: BaseModel

Parameters:
name: str
type: Type1
criteria: str
description: str | None
return_type: ReturnType | None
enabled_in_prod: bool | None
needs_ground_truth: bool | None
sampling_percentage: int | None
model_provider: str | None
model_name: str | None
scale: int | None
threshold: Threshold | None
categories: List[Dict[str, Any]] | None
child_metrics: List[Dict[str, Any]] | None
filters: Dict[str, Any] | None
id: str | None
created_at: str | None
updated_at: str | None
class honeyhive.models.generated.MetricEdit(*, metric_id, name=None, type=None, criteria=None, code_snippet=None, description=None, return_type=None, enabled_in_prod=None, needs_ground_truth=None, sampling_percentage=None, model_provider=None, model_name=None, scale=None, threshold=None, categories=None, child_metrics=None, filters=None)[source]

Bases: BaseModel

Parameters:
metric_id: str
name: str | None
type: Type1 | None
criteria: str | None
code_snippet: str | None
description: str | None
return_type: ReturnType | None
enabled_in_prod: bool | None
needs_ground_truth: bool | None
sampling_percentage: int | None
model_provider: str | None
model_name: str | None
scale: int | None
threshold: Threshold | None
categories: List[Dict[str, Any]] | None
child_metrics: List[Dict[str, Any]] | None
filters: Dict[str, Any] | None
class honeyhive.models.generated.ToolType(value)[source]

Bases: Enum

function = 'function'
tool = 'tool'
class honeyhive.models.generated.Tool(*, _id=None, task, name, description=None, parameters, tool_type)[source]

Bases: BaseModel

Parameters:
field_id: str | None
task: str
name: str
description: str | None
parameters: Dict[str, Any]
tool_type: ToolType
class honeyhive.models.generated.Type3(value)[source]

Bases: Enum

function = 'function'
tool = 'tool'
class honeyhive.models.generated.CreateToolRequest(*, task, name, description=None, parameters, type)[source]

Bases: BaseModel

Parameters:
task: str
name: str
description: str | None
parameters: Dict[str, Any]
type: Type3
class honeyhive.models.generated.UpdateToolRequest(*, id, name, description=None, parameters)[source]

Bases: BaseModel

Parameters:
id: str
name: str
description: str | None
parameters: Dict[str, Any]
class honeyhive.models.generated.Datapoint(*, _id=None, tenant=None, project_id=None, created_at=None, updated_at=None, inputs=None, history=None, ground_truth=None, linked_event=None, linked_evals=None, linked_datasets=None, saved=None, type=None, metadata=None)[source]

Bases: BaseModel

Parameters:
field_id: str | None
tenant: str | None
project_id: str | None
created_at: str | None
updated_at: str | None
inputs: Dict[str, Any] | None
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_event: str | None
linked_evals: List[str] | None
linked_datasets: List[str] | None
saved: bool | None
type: str | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.CreateDatapointRequest(*, project, inputs, history=None, ground_truth=None, linked_event=None, linked_datasets=None, metadata=None)[source]

Bases: BaseModel

Parameters:
project: str
inputs: Dict[str, Any]
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_event: str | None
linked_datasets: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.UpdateDatapointRequest(*, inputs=None, history=None, ground_truth=None, linked_evals=None, linked_datasets=None, metadata=None)[source]

Bases: BaseModel

Parameters:
inputs: Dict[str, Any] | None
history: List[Dict[str, Any]] | None
ground_truth: Dict[str, Any] | None
linked_evals: List[str] | None
linked_datasets: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.Type4(value)[source]

Bases: Enum

evaluation = 'evaluation'
fine_tuning = 'fine-tuning'
class honeyhive.models.generated.PipelineType(value)[source]

Bases: Enum

event = 'event'
session = 'session'
class honeyhive.models.generated.CreateDatasetRequest(*, project, name, description=None, type=None, datapoints=None, linked_evals=None, saved=None, pipeline_type=None, metadata=None)[source]

Bases: BaseModel

Parameters:
project: str
name: str
description: str | None
type: Type4 | None
datapoints: List[str] | None
linked_evals: List[str] | None
saved: bool | None
pipeline_type: PipelineType | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.Dataset(*, dataset_id=None, project=None, name=None, description=None, type=None, datapoints=None, num_points=None, linked_evals=None, saved=None, pipeline_type=None, created_at=None, updated_at=None, metadata=None)[source]

Bases: BaseModel

Parameters:

  • dataset_id (str | None)

  • project (str | None)

  • name (str | None)

  • description (str | None)

  • type (Type4 | None)

  • datapoints (List[str] | None)

  • num_points (int | None)

  • linked_evals (List[str] | None)

  • saved (bool | None)

  • pipeline_type (PipelineType | None)

  • created_at (str | None)

  • updated_at (str | None)

  • metadata (Dict[str, Any] | None)

dataset_id: str | None
project: str | None
name: str | None
description: str | None
type: Type4 | None
datapoints: List[str] | None
num_points: int | None
linked_evals: List[str] | None
saved: bool | None
pipeline_type: PipelineType | None
created_at: str | None
updated_at: str | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.DatasetUpdate(*, dataset_id, name=None, description=None, datapoints=None, linked_evals=None, metadata=None)[source]

Bases: BaseModel

Parameters:
dataset_id: str
name: str | None
description: str | None
datapoints: List[str] | None
linked_evals: List[str] | None
metadata: Dict[str, Any] | None
class honeyhive.models.generated.CreateProjectRequest(*, name, description=None)[source]

Bases: BaseModel

Parameters:

  • name (str)

  • description (str | None)

name: str
description: str | None
class honeyhive.models.generated.UpdateProjectRequest(*, project_id, name=None, description=None)[source]

Bases: BaseModel

Parameters:

  • project_id (str)

  • name (str | None)

  • description (str | None)

project_id: str
name: str | None
description: str | None
class honeyhive.models.generated.Project(*, id=None, name, description)[source]

Bases: BaseModel

Parameters:

  • id (str | None)

  • name (str)

  • description (str)

id: str | None
name: str
description: str
class honeyhive.models.generated.Status(value)[source]

Bases: Enum

pending = 'pending'
completed = 'completed'
class honeyhive.models.generated.UpdateRunResponse(*, evaluation=None, warning=None)[source]

Bases: BaseModel

Parameters:
evaluation: Dict[str, Any] | None
warning: str | None
class honeyhive.models.generated.Datapoints(*, passed=None, failed=None)[source]

Bases: BaseModel

Parameters:
passed: List[str] | None
failed: List[str] | None
class honeyhive.models.generated.Detail(*, metric_name=None, metric_type=None, event_name=None, event_type=None, aggregate=None, values=None, datapoints=None)[source]

Bases: BaseModel

Parameters:
metric_name: str | None
metric_type: str | None
event_name: str | None
event_type: str | None
aggregate: float | None
values: List[float | bool] | None
datapoints: Datapoints | None
class honeyhive.models.generated.Metrics(*, aggregation_function=None, details=None)[source]

Bases: BaseModel

Parameters:

  • aggregation_function (str | None)

  • details (List[Detail] | None)

aggregation_function: str | None
details: List[Detail] | None
class honeyhive.models.generated.Metric1(*, name=None, event_name=None, event_type=None, value=None, passed=None)[source]

Bases: BaseModel

Parameters:

  • name (str | None)

  • event_name (str | None)

  • event_type (str | None)

  • value (bool | float | None)

  • passed (bool | None)

name: str | None
event_name: str | None
event_type: str | None
value: float | bool | None
passed: bool | None
class honeyhive.models.generated.Datapoint1(*, datapoint_id=None, session_id=None, passed=None, metrics=None)[source]

Bases: BaseModel

Parameters:
datapoint_id: str | None
session_id: str | None
passed: bool | None
metrics: List[Metric1] | None
class honeyhive.models.generated.ExperimentResultResponse(*, status=None, success=None, passed=None, failed=None, metrics=None, datapoints=None)[source]

Bases: BaseModel

Parameters:
status: str | None
success: bool | None
passed: List[str] | None
failed: List[str] | None
metrics: Metrics | None
datapoints: List[Datapoint1] | None
class honeyhive.models.generated.Metric2(*, metric_name=None, event_name=None, metric_type=None, event_type=None, old_aggregate=None, new_aggregate=None, found_count=None, improved_count=None, degraded_count=None, same_count=None, improved=None, degraded=None, same=None, old_values=None, new_values=None)[source]

Bases: BaseModel

Parameters:

  • metric_name (str | None)

  • event_name (str | None)

  • metric_type (str | None)

  • event_type (str | None)

  • old_aggregate (float | None)

  • new_aggregate (float | None)

  • found_count (int | None)

  • improved_count (int | None)

  • degraded_count (int | None)

  • same_count (int | None)

  • improved (List[str] | None)

  • degraded (List[str] | None)

  • same (List[str] | None)

  • old_values (List[float | bool] | None)

  • new_values (List[float | bool] | None)

metric_name: str | None
event_name: str | None
metric_type: str | None
event_type: str | None
old_aggregate: float | None
new_aggregate: float | None
found_count: int | None
improved_count: int | None
degraded_count: int | None
same_count: int | None
improved: List[str] | None
degraded: List[str] | None
same: List[str] | None
old_values: List[float | bool] | None
new_values: List[float | bool] | None
class honeyhive.models.generated.EventDetail(*, event_name=None, event_type=None, presence=None)[source]

Bases: BaseModel

Parameters:

  • event_name (str | None)

  • event_type (str | None)

  • presence (str | None)

event_name: str | None
event_type: str | None
presence: str | None
class honeyhive.models.generated.OldRun(*, _id=None, run_id=None, project=None, tenant=None, created_at=None, event_ids=None, session_ids=None, dataset_id=None, datapoint_ids=None, evaluators=None, results=None, configuration=None, metadata=None, passing_ranges=None, status=None, name=None)[source]

Bases: BaseModel

Parameters:
field_id: str | None
run_id: str | None
project: str | None
tenant: str | None
created_at: AwareDatetime | None
event_ids: List[str] | None
session_ids: List[str] | None
dataset_id: str | None
datapoint_ids: List[str] | None
evaluators: List[Dict[str, Any]] | None
results: Dict[str, Any] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
passing_ranges: Dict[str, Any] | None
status: str | None
name: str | None
class honeyhive.models.generated.NewRun(*, _id=None, run_id=None, project=None, tenant=None, created_at=None, event_ids=None, session_ids=None, dataset_id=None, datapoint_ids=None, evaluators=None, results=None, configuration=None, metadata=None, passing_ranges=None, status=None, name=None)[source]

Bases: BaseModel

Parameters:
field_id: str | None
run_id: str | None
project: str | None
tenant: str | None
created_at: AwareDatetime | None
event_ids: List[str] | None
session_ids: List[str] | None
dataset_id: str | None
datapoint_ids: List[str] | None
evaluators: List[Dict[str, Any]] | None
results: Dict[str, Any] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
passing_ranges: Dict[str, Any] | None
status: str | None
name: str | None
class honeyhive.models.generated.ExperimentComparisonResponse(*, metrics=None, commonDatapoints=None, event_details=None, old_run=None, new_run=None)[source]

Bases: BaseModel

Parameters:
metrics: List[Metric2] | None
commonDatapoints: List[str] | None
event_details: List[EventDetail] | None
old_run: OldRun | None
new_run: NewRun | None
class honeyhive.models.generated.UUIDType(root=PydanticUndefined)[source]

Bases: RootModel[UUID]

Parameters:

root (UUID)

root: UUID
class honeyhive.models.generated.EnvEnum(value)[source]

Bases: Enum

dev = 'dev'
staging = 'staging'
prod = 'prod'
class honeyhive.models.generated.CallType(value)[source]

Bases: Enum

chat = 'chat'
completion = 'completion'
class honeyhive.models.generated.SelectedFunction(*, id=None, name=None, description=None, parameters=None)[source]

Bases: BaseModel

Parameters:

  • id (str | None)

  • name (str | None)

  • description (str | None)

  • parameters (Dict[str, Any] | None)

id: str | None
name: str | None
description: str | None
parameters: Dict[str, Any] | None
class honeyhive.models.generated.FunctionCallParams(value)[source]

Bases: Enum

none = 'none'
auto = 'auto'
force = 'force'
class honeyhive.models.generated.Parameters(*, call_type, model, hyperparameters=None, responseFormat=None, selectedFunctions=None, functionCallParams=None, forceFunction=None, **extra_data)[source]

Bases: BaseModel

Parameters:
call_type: CallType
model: str
hyperparameters: Dict[str, Any] | None
responseFormat: Dict[str, Any] | None
selectedFunctions: List[SelectedFunction] | None
functionCallParams: FunctionCallParams | None
forceFunction: Dict[str, Any] | None
class honeyhive.models.generated.Type6(value)[source]

Bases: Enum

LLM = 'LLM'
pipeline = 'pipeline'
class honeyhive.models.generated.Configuration(*, _id=None, project, name, env=None, provider, parameters, type=None, user_properties=None)[source]

Bases: BaseModel

Parameters:
field_id: str | None
project: str
name: str
env: List[EnvEnum] | None
provider: str
parameters: Parameters
type: Type6 | None
user_properties: Dict[str, Any] | None
class honeyhive.models.generated.Parameters1(*, call_type, model, hyperparameters=None, responseFormat=None, selectedFunctions=None, functionCallParams=None, forceFunction=None, **extra_data)[source]

Bases: BaseModel

Parameters:
call_type: CallType
model: str
hyperparameters: Dict[str, Any] | None
responseFormat: Dict[str, Any] | None
selectedFunctions: List[SelectedFunction] | None
functionCallParams: FunctionCallParams | None
forceFunction: Dict[str, Any] | None
class honeyhive.models.generated.PutConfigurationRequest(*, project, name, provider, parameters, env=None, type=None, user_properties=None)[source]

Bases: BaseModel

Parameters:
project: str
name: str
provider: str
parameters: Parameters1
env: List[EnvEnum] | None
type: Type6 | None
user_properties: Dict[str, Any] | None
class honeyhive.models.generated.Parameters2(*, call_type, model, hyperparameters=None, responseFormat=None, selectedFunctions=None, functionCallParams=None, forceFunction=None, **extra_data)[source]

Bases: BaseModel

Parameters:
call_type: CallType
model: str
hyperparameters: Dict[str, Any] | None
responseFormat: Dict[str, Any] | None
selectedFunctions: List[SelectedFunction] | None
functionCallParams: FunctionCallParams | None
forceFunction: Dict[str, Any] | None
class honeyhive.models.generated.PostConfigurationRequest(*, project, name, provider, parameters, env=None, user_properties=None)[source]

Bases: BaseModel

Parameters:
project: str
name: str
provider: str
parameters: Parameters2
env: List[EnvEnum] | None
user_properties: Dict[str, Any] | None
class honeyhive.models.generated.CreateRunRequest(*, project, name, event_ids, dataset_id=None, datapoint_ids=None, configuration=None, metadata=None, status=None)[source]

Bases: BaseModel

Parameters:
project: str
name: str
event_ids: List[UUIDType]
dataset_id: str | None
datapoint_ids: List[str] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
status: Status | None
class honeyhive.models.generated.UpdateRunRequest(*, event_ids=None, dataset_id=None, datapoint_ids=None, configuration=None, metadata=None, name=None, status=None)[source]

Bases: BaseModel

Parameters:
event_ids: List[UUIDType] | None
dataset_id: str | None
datapoint_ids: List[str] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
name: str | None
status: Status | None
class honeyhive.models.generated.DeleteRunResponse(*, id=None, deleted=None)[source]

Bases: BaseModel

Parameters:
id: UUIDType | None
deleted: bool | None
class honeyhive.models.generated.EvaluationRun(*, run_id=None, project=None, created_at=None, event_ids=None, dataset_id=None, datapoint_ids=None, results=None, configuration=None, metadata=None, status=None, name=None)[source]

Bases: BaseModel

Parameters:
run_id: UUIDType | None
project: str | None
created_at: AwareDatetime | None
event_ids: List[UUIDType] | None
dataset_id: str | None
datapoint_ids: List[str] | None
results: Dict[str, Any] | None
configuration: Dict[str, Any] | None
metadata: Dict[str, Any] | None
status: Status | None
name: str | None
class honeyhive.models.generated.CreateRunResponse(*, evaluation=None, run_id=None)[source]

Bases: BaseModel

Parameters:
evaluation: EvaluationRun | None
run_id: UUIDType | None
class honeyhive.models.generated.GetRunsResponse(*, evaluations=None)[source]

Bases: BaseModel

Parameters:

evaluations (List[EvaluationRun] | None)

evaluations: List[EvaluationRun] | None
class honeyhive.models.generated.GetRunResponse(*, evaluation=None)[source]

Bases: BaseModel

Parameters:

evaluation (EvaluationRun | None)

evaluation: EvaluationRun | None

Note

Key Models Included:

Request Models: - CreateRunRequest - Create experiment runs - CreateDatasetRequest - Create datasets - CreateProjectRequest - Create projects - CreateToolRequest - Create tools - UpdateRunRequest, UpdateProjectRequest, UpdateToolRequest - Update operations

Response Models: - CreateRunResponse - Run creation response - Dataset - Dataset information - DeleteRunResponse - Deletion confirmation - GetRunResponse, GetRunsResponse - Run retrieval - NewRun, OldRun - Run comparison models

Supporting Models: - SessionStartRequest, SessionPropertiesBatch - Session management - ExperimentComparisonResponse, ExperimentResultResponse - Experiment results - FunctionCallParams, SelectedFunction, Parameters - Configuration - Metric1, Metric2, MetricEdit - Metrics - Threshold, Operator - Evaluation criteria

Enums: - CallType - LLM call types (chat, completion) - EnvEnum - Environments (dev, staging, prod) - PipelineType - Pipeline types (event, session) - ToolType, ReturnType - Tool and return type specifications - Type1, Type3, Type4, Type6 - Type categorizations - UUIDType - UUID handling

Configuration Models

ServerURLMixin

class honeyhive.config.models.base.ServerURLMixin[source]

Bases: object

Mixin for server URL configuration with HH_API_URL environment variable support.

This mixin provides the server_url field with proper environment variable loading for classes that need to support custom HoneyHive server URLs. It can be used by both APIClientConfig and TracerConfig to avoid field duplication.

Environment Variables:

HH_API_URL: Custom HoneyHive server URL

Examples

>>> class MyConfig(BaseHoneyHiveConfig, ServerURLMixin):
...     pass
>>> config = MyConfig()  # Loads from HH_API_URL if set
server_url: str = FieldInfo(annotation=NoneType, required=False, default='https://api.honeyhive.ai', alias_priority=2, validation_alias=AliasChoices(choices=['HH_API_URL', 'server_url']), description='Custom HoneyHive server URL', examples=['https://api.honeyhive.ai', 'https://custom.honeyhive.com'])
classmethod validate_server_url(v)[source]

Validate server URL format with graceful degradation.

Parameters:

v (Any) – The server URL to validate

Returns:

The validated and normalized server URL, or default if invalid

Return type:

str

Experiment Models

ExperimentRunStatus

class honeyhive.experiments.models.ExperimentRunStatus(value)[source]

Bases: str, Enum

Extended status enum with all backend values.

The generated Status enum only includes ‘pending’ and ‘completed’, but the backend supports additional states.

PENDING = 'pending'
COMPLETED = 'completed'
RUNNING = 'running'
FAILED = 'failed'
CANCELLED = 'cancelled'

RunComparisonResult

class honeyhive.experiments.models.RunComparisonResult(*, new_run_id, old_run_id, common_datapoints, new_only_datapoints=0, old_only_datapoints=0, metric_deltas=<factory>)[source]

Bases: BaseModel

Comparison between two experiment runs.

This model represents the delta analysis between a new run and an old run, including metric changes and datapoint differences.

Retrieved from: GET /runs/:new_run_id/compare-with/:old_run_id

Parameters:

  • new_run_id (str)

  • old_run_id (str)

  • common_datapoints (int)

  • new_only_datapoints (int)

  • old_only_datapoints (int)

  • metric_deltas (Dict[str, Any])

new_run_id: str
old_run_id: str
common_datapoints: int
new_only_datapoints: int
old_only_datapoints: int
metric_deltas: Dict[str, Any]
get_metric_delta(metric_name)[source]

Get delta information for a specific metric.

Parameters:

metric_name (str) – Name of the metric

Returns:

Delta information including new_value, old_value, delta, percent_change

Return type:

Dict[str, Any] | None

Example

>>> comparison.get_metric_delta("accuracy")
{
    'new_value': 0.85,
    'old_value': 0.80,
    'delta': 0.05,
    'percent_change': 6.25
}
list_improved_metrics()[source]

List metrics that improved in the new run.

Returns:

List of metric names where improved_count > 0

Return type:

List[str]

list_degraded_metrics()[source]

List metrics that degraded in the new run.

Returns:

List of metric names where degraded_count > 0

Return type:

List[str]

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

ExperimentContext

class honeyhive.experiments.core.ExperimentContext(run_id, dataset_id, project, *, run_name=None, source='evaluation', metadata=None)[source]

Bases: object

Lightweight experiment context for metadata linking.

NOTE: This is NOT a replacement for tracer config. This is just a convenience class for organizing experiment metadata that gets passed to the tracer.

The tracer handles actual metadata propagation when is_evaluation=True.

Parameters:
run_id

Experiment run identifier

dataset_id

Dataset identifier (may have EXT- prefix)

project

Project identifier

source

Source identifier (default: “evaluation”)

metadata

Additional metadata dictionary

Example

>>> context = ExperimentContext(
...     run_id="run-123",
...     dataset_id="EXT-abc",
...     project="my-project"
... )
>>> tracer_config = context.to_tracer_config("dp-1")
>>> tracer_config["is_evaluation"]
True
to_tracer_config(datapoint_id)[source]

Convert to tracer initialization config.

This returns kwargs for HoneyHiveTracer(…) initialization. The tracer will automatically propagate all metadata to spans when is_evaluation=True.

Parameters:

datapoint_id (str) – Datapoint identifier for this execution

Returns:

Dictionary of tracer initialization kwargs

Return type:

Dict[str, Any]

Example

>>> config = context.to_tracer_config("dp-1")
>>> config
{
    'project': 'my-project',
    'is_evaluation': True,
    'run_id': 'run-123',
    'dataset_id': 'EXT-abc',
    'datapoint_id': 'dp-1',
    'source': 'evaluation'
}

See Also