API Client Classes

This section documents all API client classes for interacting with the HoneyHive platform.

HoneyHive Client

The main client class for interacting with the HoneyHive API.

class honeyhive.api.client.HoneyHive(*, api_key=None, server_url=None, timeout=None, retry_config=None, rate_limit_calls=100, rate_limit_window=60.0, max_connections=10, max_keepalive=20, test_mode=None, verbose=False, tracer_instance=None)[source]

Bases: object

Main HoneyHive API client.

Parameters:
  • api_key (str | None)

  • server_url (str | None)

  • timeout (float | None)

  • retry_config (RetryConfig | None)

  • rate_limit_calls (int)

  • rate_limit_window (float)

  • max_connections (int)

  • max_keepalive (int)

  • test_mode (bool | None)

  • verbose (bool)

  • tracer_instance (Any | None)

__init__(*, api_key=None, server_url=None, timeout=None, retry_config=None, rate_limit_calls=100, rate_limit_window=60.0, max_connections=10, max_keepalive=20, test_mode=None, verbose=False, tracer_instance=None)[source]

Initialize the HoneyHive client.

Parameters:
  • api_key (str | None) – API key for authentication

  • server_url (str | None) – Server URL for the API

  • timeout (float | None) – Request timeout in seconds

  • retry_config (RetryConfig | None) – Retry configuration

  • rate_limit_calls (int) – Maximum calls per time window

  • rate_limit_window (float) – Time window in seconds

  • max_connections (int) – Maximum connections in pool

  • max_keepalive (int) – Maximum keepalive connections

  • test_mode (bool | None) – Enable test mode (None = use config default)

  • verbose (bool) – Enable verbose logging for API debugging

  • tracer_instance (Any | None) – Optional tracer instance for multi-instance logging

logger: HoneyHiveLogger | None
property client_kwargs: Dict[str, Any]

Get common client configuration.

property sync_client: Client

Get or create sync HTTP client.

property async_client: AsyncClient

Get or create async HTTP client.

get_health()[source]

Get API health status. Returns basic info since health endpoint may not exist.

Return type:

Dict[str, Any]

async get_health_async()[source]

Get API health status asynchronously. Returns basic info since health endpoint may not exist.

Return type:

Dict[str, Any]

request(method, path, params=None, json=None, **kwargs)[source]

Make a synchronous HTTP request with rate limiting and retry logic.

Parameters:
Return type:

Response

async request_async(method, path, params=None, json=None, **kwargs)[source]

Make an asynchronous HTTP request with rate limiting and retry logic.

Parameters:
Return type:

Response

close()[source]

Close the HTTP clients.

Return type:

None

async aclose()[source]

Close the HTTP clients asynchronously.

Return type:

None

Usage Example

from honeyhive import HoneyHive as Client

# Initialize the client
client = honeyhive.HoneyHive(
    api_key="your-api-key",
    project="your-project"
)

# Access API endpoints
datasets = client.datasets.list_datasets(project="your-project")
metrics = client.metrics.get_metrics(project="your-project")

RateLimiter

Rate limiting for API calls to prevent exceeding rate limits.

class honeyhive.api.client.RateLimiter(max_calls=100, time_window=60.0)[source]

Bases: object

Simple rate limiter for API calls.

Provides basic rate limiting functionality to prevent exceeding API rate limits.

Parameters:
__init__(max_calls=100, time_window=60.0)[source]

Initialize the rate limiter.

Parameters:
  • max_calls (int) – Maximum number of calls allowed in the time window

  • time_window (float) – Time window in seconds for rate limiting

can_call()[source]

Check if a call can be made.

Returns:

True if a call can be made, False if rate limit is exceeded

Return type:

bool

wait_if_needed()[source]

Wait if rate limit is exceeded.

Blocks execution until a call can be made.

Return type:

None

Example

from honeyhive.api.client import RateLimiter

# Create rate limiter (100 calls per 60 seconds)
limiter = RateLimiter(max_calls=100, time_window=60.0)

# Check if call is allowed
if limiter.can_call():
    # Make API call
    pass

# Or wait automatically
limiter.wait_if_needed()
# Make API call

BaseAPI

Base class for all API endpoint clients.

class honeyhive.api.base.BaseAPI(client)[source]

Bases: object

Base class for all API modules.

Parameters:

client (HoneyHive)

__init__(client)[source]

Initialize the API module with a client.

Parameters:

client (HoneyHive) – HoneyHive client instance

DatasetsAPI

API client for dataset operations.

Recent Updates: Enhanced filtering capabilities for list_datasets() including name and include_datapoints parameters. See method documentation below for details.

class honeyhive.api.datasets.DatasetsAPI(client)[source]

Bases: BaseAPI

API for dataset operations.

Parameters:

client (HoneyHive)

create_dataset(request)[source]

Create a new dataset using CreateDatasetRequest model.

Parameters:

request (CreateDatasetRequest)

Return type:

Dataset

create_dataset_from_dict(dataset_data)[source]

Create a new dataset from dictionary (legacy method).

Parameters:

dataset_data (dict)

Return type:

Dataset

async create_dataset_async(request)[source]

Create a new dataset asynchronously using CreateDatasetRequest model.

Parameters:

request (CreateDatasetRequest)

Return type:

Dataset

async create_dataset_from_dict_async(dataset_data)[source]

Create a new dataset asynchronously from dictionary (legacy method).

Parameters:

dataset_data (dict)

Return type:

Dataset

get_dataset(dataset_id)[source]

Get a dataset by ID.

Parameters:

dataset_id (str)

Return type:

Dataset

async get_dataset_async(dataset_id)[source]

Get a dataset by ID asynchronously.

Parameters:

dataset_id (str)

Return type:

Dataset

list_datasets(project=None, *, dataset_type=None, dataset_id=None, name=None, include_datapoints=False, limit=100)[source]

List datasets with optional filtering.

Parameters:
  • project (str | None) – Project name to filter by

  • dataset_type (Literal['evaluation', 'fine-tuning'] | None) – Type of dataset - “evaluation” or “fine-tuning”

  • dataset_id (str | None) – Specific dataset ID to filter by

  • name (str | None) – Dataset name to filter by (exact match)

  • include_datapoints (bool) – Include datapoints in response (may impact performance)

  • limit (int) – Maximum number of datasets to return (default: 100)

Returns:

List of Dataset objects matching the filters

Return type:

List[Dataset]

Examples

Find dataset by name:

datasets = client.datasets.list_datasets(
    project="My Project",
    name="Training Data Q4"
)

Get specific dataset with datapoints:

dataset = client.datasets.list_datasets(
    dataset_id="663876ec4611c47f4970f0c3",
    include_datapoints=True
)[0]

Filter by type and name:

eval_datasets = client.datasets.list_datasets(
    dataset_type="evaluation",
    name="Regression Tests"
)
async list_datasets_async(project=None, *, dataset_type=None, dataset_id=None, name=None, include_datapoints=False, limit=100)[source]

List datasets asynchronously with optional filtering.

Parameters:
  • project (str | None) – Project name to filter by

  • dataset_type (Literal['evaluation', 'fine-tuning'] | None) – Type of dataset - “evaluation” or “fine-tuning”

  • dataset_id (str | None) – Specific dataset ID to filter by

  • name (str | None) – Dataset name to filter by (exact match)

  • include_datapoints (bool) – Include datapoints in response (may impact performance)

  • limit (int) – Maximum number of datasets to return (default: 100)

Returns:

List of Dataset objects matching the filters

Return type:

List[Dataset]

Examples

Find dataset by name:

datasets = await client.datasets.list_datasets_async(
    project="My Project",
    name="Training Data Q4"
)

Get specific dataset with datapoints:

dataset = await client.datasets.list_datasets_async(
    dataset_id="663876ec4611c47f4970f0c3",
    include_datapoints=True
)

Filter by type and name:

eval_datasets = await client.datasets.list_datasets_async(
    dataset_type="evaluation",
    name="Regression Tests"
)
update_dataset(dataset_id, request)[source]

Update a dataset using DatasetUpdate model.

Parameters:
Return type:

Dataset

update_dataset_from_dict(dataset_id, dataset_data)[source]

Update a dataset from dictionary (legacy method).

Parameters:
  • dataset_id (str)

  • dataset_data (dict)

Return type:

Dataset

async update_dataset_async(dataset_id, request)[source]

Update a dataset asynchronously using DatasetUpdate model.

Parameters:
Return type:

Dataset

async update_dataset_from_dict_async(dataset_id, dataset_data)[source]

Update a dataset asynchronously from dictionary (legacy method).

Parameters:
  • dataset_id (str)

  • dataset_data (dict)

Return type:

Dataset

delete_dataset(dataset_id)[source]

Delete a dataset by ID.

Parameters:

dataset_id (str)

Return type:

bool

async delete_dataset_async(dataset_id)[source]

Delete a dataset by ID asynchronously.

Parameters:

dataset_id (str)

Return type:

bool

Methods

create_dataset

DatasetsAPI.create_dataset(request)[source]

Create a new dataset using CreateDatasetRequest model.

Parameters:

request (CreateDatasetRequest)

Return type:

Dataset

create_dataset_async

async DatasetsAPI.create_dataset_async(request)[source]

Create a new dataset asynchronously using CreateDatasetRequest model.

Parameters:

request (CreateDatasetRequest)

Return type:

Dataset

list_datasets

DatasetsAPI.list_datasets(project=None, *, dataset_type=None, dataset_id=None, name=None, include_datapoints=False, limit=100)[source]

List datasets with optional filtering.

Parameters:
  • project (str | None) – Project name to filter by

  • dataset_type (Literal['evaluation', 'fine-tuning'] | None) – Type of dataset - “evaluation” or “fine-tuning”

  • dataset_id (str | None) – Specific dataset ID to filter by

  • name (str | None) – Dataset name to filter by (exact match)

  • include_datapoints (bool) – Include datapoints in response (may impact performance)

  • limit (int) – Maximum number of datasets to return (default: 100)

Returns:

List of Dataset objects matching the filters

Return type:

List[Dataset]

Examples

Find dataset by name:

datasets = client.datasets.list_datasets(
    project="My Project",
    name="Training Data Q4"
)

Get specific dataset with datapoints:

dataset = client.datasets.list_datasets(
    dataset_id="663876ec4611c47f4970f0c3",
    include_datapoints=True
)[0]

Filter by type and name:

eval_datasets = client.datasets.list_datasets(
    dataset_type="evaluation",
    name="Regression Tests"
)

get_dataset

DatasetsAPI.get_dataset(dataset_id)[source]

Get a dataset by ID.

Parameters:

dataset_id (str)

Return type:

Dataset

update_dataset

DatasetsAPI.update_dataset(dataset_id, request)[source]

Update a dataset using DatasetUpdate model.

Parameters:
Return type:

Dataset

delete_dataset

DatasetsAPI.delete_dataset(dataset_id)[source]

Delete a dataset by ID.

Parameters:

dataset_id (str)

Return type:

bool

Example

from honeyhive import HoneyHive as Client
from honeyhive.models import CreateDatasetRequest

client = honeyhive.HoneyHive(api_key="your-api-key")

# Create a dataset
dataset = client.datasets.create_dataset(
    CreateDatasetRequest(
        project="your-project",
        name="test-dataset",
        description="Test dataset for evaluation"
    )
)

# List datasets
datasets = client.datasets.list_datasets(project="your-project")

# Get specific dataset
dataset = client.datasets.get_dataset(dataset_id="dataset-id")

MetricsAPI

API client for metrics operations.

class honeyhive.api.metrics.MetricsAPI(client)[source]

Bases: BaseAPI

API for metric operations.

Parameters:

client (HoneyHive)

create_metric(request)[source]

Create a new metric using Metric model.

Parameters:

request (Metric)

Return type:

Metric

create_metric_from_dict(metric_data)[source]

Create a new metric from dictionary (legacy method).

Parameters:

metric_data (dict)

Return type:

Metric

async create_metric_async(request)[source]

Create a new metric asynchronously using Metric model.

Parameters:

request (Metric)

Return type:

Metric

async create_metric_from_dict_async(metric_data)[source]

Create a new metric asynchronously from dictionary (legacy method).

Parameters:

metric_data (dict)

Return type:

Metric

get_metric(metric_id)[source]

Get a metric by ID.

Parameters:

metric_id (str)

Return type:

Metric

async get_metric_async(metric_id)[source]

Get a metric by ID asynchronously.

Parameters:

metric_id (str)

Return type:

Metric

list_metrics(project=None, limit=100)[source]

List metrics with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

List[Metric]

async list_metrics_async(project=None, limit=100)[source]

List metrics asynchronously with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

List[Metric]

update_metric(metric_id, request)[source]

Update a metric using MetricEdit model.

Parameters:
Return type:

Metric

update_metric_from_dict(metric_id, metric_data)[source]

Update a metric from dictionary (legacy method).

Parameters:
  • metric_id (str)

  • metric_data (dict)

Return type:

Metric

async update_metric_async(metric_id, request)[source]

Update a metric asynchronously using MetricEdit model.

Parameters:
Return type:

Metric

async update_metric_from_dict_async(metric_id, metric_data)[source]

Update a metric asynchronously from dictionary (legacy method).

Parameters:
  • metric_id (str)

  • metric_data (dict)

Return type:

Metric

delete_metric(metric_id)[source]

Delete a metric by ID.

Note: Deleting metrics via API is not authorized for security reasons. Please use the HoneyHive web application to delete metrics.

Parameters:

metric_id (str) – The ID of the metric to delete

Raises:

AuthenticationError – Always raised as this operation is not permitted via API

Return type:

bool

async delete_metric_async(metric_id)[source]

Delete a metric by ID asynchronously.

Note: Deleting metrics via API is not authorized for security reasons. Please use the HoneyHive web application to delete metrics.

Parameters:

metric_id (str) – The ID of the metric to delete

Raises:

AuthenticationError – Always raised as this operation is not permitted via API

Return type:

bool

Example

from honeyhive import HoneyHive as Client

client = honeyhive.HoneyHive(api_key="your-api-key")

# Get metrics for a project
metrics = client.metrics.get_metrics(
    project="your-project",
    start_time="2024-01-01T00:00:00Z",
    end_time="2024-01-31T23:59:59Z"
)

ProjectsAPI

API client for project operations.

class honeyhive.api.projects.ProjectsAPI(client)[source]

Bases: BaseAPI

API for project operations.

Parameters:

client (HoneyHive)

create_project(request)[source]

Create a new project using CreateProjectRequest model.

Parameters:

request (CreateProjectRequest)

Return type:

Project

create_project_from_dict(project_data)[source]

Create a new project from dictionary (legacy method).

Parameters:

project_data (dict)

Return type:

Project

async create_project_async(request)[source]

Create a new project asynchronously using CreateProjectRequest model.

Parameters:

request (CreateProjectRequest)

Return type:

Project

async create_project_from_dict_async(project_data)[source]

Create a new project asynchronously from dictionary (legacy method).

Parameters:

project_data (dict)

Return type:

Project

get_project(project_id)[source]

Get a project by ID.

Parameters:

project_id (str)

Return type:

Project

async get_project_async(project_id)[source]

Get a project by ID asynchronously.

Parameters:

project_id (str)

Return type:

Project

list_projects(limit=100)[source]

List projects with optional filtering.

Parameters:

limit (int)

Return type:

List[Project]

async list_projects_async(limit=100)[source]

List projects asynchronously with optional filtering.

Parameters:

limit (int)

Return type:

List[Project]

update_project(project_id, request)[source]

Update a project using UpdateProjectRequest model.

Parameters:
Return type:

Project

update_project_from_dict(project_id, project_data)[source]

Update a project from dictionary (legacy method).

Parameters:
  • project_id (str)

  • project_data (dict)

Return type:

Project

async update_project_async(project_id, request)[source]

Update a project asynchronously using UpdateProjectRequest model.

Parameters:
Return type:

Project

async update_project_from_dict_async(project_id, project_data)[source]

Update a project asynchronously from dictionary (legacy method).

Parameters:
  • project_id (str)

  • project_data (dict)

Return type:

Project

delete_project(project_id)[source]

Delete a project by ID.

Parameters:

project_id (str)

Return type:

bool

async delete_project_async(project_id)[source]

Delete a project by ID asynchronously.

Parameters:

project_id (str)

Return type:

bool

Methods

create_project

ProjectsAPI.create_project(request)[source]

Create a new project using CreateProjectRequest model.

Parameters:

request (CreateProjectRequest)

Return type:

Project

list_projects

ProjectsAPI.list_projects(limit=100)[source]

List projects with optional filtering.

Parameters:

limit (int)

Return type:

List[Project]

get_project

ProjectsAPI.get_project(project_id)[source]

Get a project by ID.

Parameters:

project_id (str)

Return type:

Project

update_project

ProjectsAPI.update_project(project_id, request)[source]

Update a project using UpdateProjectRequest model.

Parameters:
Return type:

Project

delete_project

ProjectsAPI.delete_project(project_id)[source]

Delete a project by ID.

Parameters:

project_id (str)

Return type:

bool

Example

from honeyhive import HoneyHive as Client
from honeyhive.models import CreateProjectRequest

client = honeyhive.HoneyHive(api_key="your-api-key")

# Create a project
project = client.projects.create_project(
    CreateProjectRequest(
        name="my-llm-project",
        description="Production LLM application"
    )
)

# List all projects
projects = client.projects.list_projects()

SessionAPI

API client for session operations.

class honeyhive.api.session.SessionAPI(client)[source]

Bases: BaseAPI

API for session operations.

Parameters:

client (HoneyHive)

create_session(session)[source]

Create a new session using SessionStartRequest model.

Parameters:

session (SessionStartRequest)

Return type:

SessionStartResponse

create_session_from_dict(session_data)[source]

Create a new session from session data dictionary (legacy method).

Parameters:

session_data (dict)

Return type:

SessionStartResponse

async create_session_async(session)[source]

Create a new session asynchronously using SessionStartRequest model.

Parameters:

session (SessionStartRequest)

Return type:

SessionStartResponse

async create_session_from_dict_async(session_data)[source]

Create a new session asynchronously from session data dictionary (legacy method).

Parameters:

session_data (dict)

Return type:

SessionStartResponse

start_session(project, session_name, source, session_id=None, **kwargs)[source]

Start a new session using SessionStartRequest model.

Parameters:
  • project (str)

  • session_name (str)

  • source (str)

  • session_id (str | None)

  • kwargs (Any)

Return type:

SessionStartResponse

async start_session_async(project, session_name, source, session_id=None, **kwargs)[source]

Start a new session asynchronously using SessionStartRequest model.

Parameters:
  • project (str)

  • session_name (str)

  • source (str)

  • session_id (str | None)

  • kwargs (Any)

Return type:

SessionStartResponse

get_session(session_id)[source]

Get a session by ID.

Parameters:

session_id (str)

Return type:

SessionResponse

async get_session_async(session_id)[source]

Get a session by ID asynchronously.

Parameters:

session_id (str)

Return type:

SessionResponse

delete_session(session_id)[source]

Delete a session by ID.

Parameters:

session_id (str)

Return type:

bool

async delete_session_async(session_id)[source]

Delete a session by ID asynchronously.

Parameters:

session_id (str)

Return type:

bool

SessionResponse

Response model for session operations.

class honeyhive.api.session.SessionResponse(event)[source]

Bases: object

Response from getting a session.

Contains the session data retrieved from the API.

Parameters:

event (Event)

SessionStartResponse

Response model for session start operations.

class honeyhive.api.session.SessionStartResponse(session_id)[source]

Bases: object

Response from starting a session.

Contains the result of a session creation operation including the session ID.

Parameters:

session_id (str)

property id: str

Alias for session_id for compatibility.

Returns:

The session ID

Example

from honeyhive import HoneyHive as Client

client = honeyhive.HoneyHive(api_key="your-api-key")

# Start a session
session = client.session.start_session(
    project="your-project",
    session_name="user-interaction",
    metadata={"user_id": "123"}
)

# End the session
client.session.end_session(
    session_id=session.session_id,
    status="completed"
)

ToolsAPI

API client for tool operations.

class honeyhive.api.tools.ToolsAPI(client)[source]

Bases: BaseAPI

API for tool operations.

Parameters:

client (HoneyHive)

create_tool(request)[source]

Create a new tool using CreateToolRequest model.

Parameters:

request (CreateToolRequest)

Return type:

Tool

create_tool_from_dict(tool_data)[source]

Create a new tool from dictionary (legacy method).

Parameters:

tool_data (dict)

Return type:

Tool

async create_tool_async(request)[source]

Create a new tool asynchronously using CreateToolRequest model.

Parameters:

request (CreateToolRequest)

Return type:

Tool

async create_tool_from_dict_async(tool_data)[source]

Create a new tool asynchronously from dictionary (legacy method).

Parameters:

tool_data (dict)

Return type:

Tool

get_tool(tool_id)[source]

Get a tool by ID.

Parameters:

tool_id (str)

Return type:

Tool

async get_tool_async(tool_id)[source]

Get a tool by ID asynchronously.

Parameters:

tool_id (str)

Return type:

Tool

list_tools(project=None, limit=100)[source]

List tools with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

List[Tool]

async list_tools_async(project=None, limit=100)[source]

List tools asynchronously with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

List[Tool]

update_tool(tool_id, request)[source]

Update a tool using UpdateToolRequest model.

Parameters:
Return type:

Tool

update_tool_from_dict(tool_id, tool_data)[source]

Update a tool from dictionary (legacy method).

Parameters:
Return type:

Tool

async update_tool_async(tool_id, request)[source]

Update a tool asynchronously using UpdateToolRequest model.

Parameters:
Return type:

Tool

async update_tool_from_dict_async(tool_id, tool_data)[source]

Update a tool asynchronously from dictionary (legacy method).

Parameters:
Return type:

Tool

delete_tool(tool_id)[source]

Delete a tool by ID.

Parameters:

tool_id (str)

Return type:

bool

async delete_tool_async(tool_id)[source]

Delete a tool by ID asynchronously.

Parameters:

tool_id (str)

Return type:

bool

Methods

create_tool

ToolsAPI.create_tool(request)[source]

Create a new tool using CreateToolRequest model.

Parameters:

request (CreateToolRequest)

Return type:

Tool

list_tools

ToolsAPI.list_tools(project=None, limit=100)[source]

List tools with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

List[Tool]

get_tool

ToolsAPI.get_tool(tool_id)[source]

Get a tool by ID.

Parameters:

tool_id (str)

Return type:

Tool

update_tool

ToolsAPI.update_tool(tool_id, request)[source]

Update a tool using UpdateToolRequest model.

Parameters:
Return type:

Tool

delete_tool

ToolsAPI.delete_tool(tool_id)[source]

Delete a tool by ID.

Parameters:

tool_id (str)

Return type:

bool

Example

from honeyhive import HoneyHive as Client
from honeyhive.models import CreateToolRequest

client = honeyhive.HoneyHive(api_key="your-api-key")

# Create a tool
tool = client.tools.create_tool(
    CreateToolRequest(
        project="your-project",
        name="calculator",
        description="Performs mathematical calculations",
        parameters={
            "type": "object",
            "properties": {
                "operation": {"type": "string"},
                "a": {"type": "number"},
                "b": {"type": "number"}
            }
        }
    )
)

EvaluationsAPI

API client for evaluation operations.

class honeyhive.api.evaluations.EvaluationsAPI(client)[source]

Bases: BaseAPI

API client for HoneyHive evaluations.

Parameters:

client (HoneyHive)

create_run(request)[source]

Create a new evaluation run using CreateRunRequest model.

Parameters:

request (CreateRunRequest)

Return type:

CreateRunResponse

create_run_from_dict(run_data)[source]

Create a new evaluation run from dictionary (legacy method).

Parameters:

run_data (dict)

Return type:

CreateRunResponse

async create_run_async(request)[source]

Create a new evaluation run asynchronously using CreateRunRequest model.

Parameters:

request (CreateRunRequest)

Return type:

CreateRunResponse

async create_run_from_dict_async(run_data)[source]

Create a new evaluation run asynchronously from dictionary (legacy method).

Parameters:

run_data (dict)

Return type:

CreateRunResponse

get_run(run_id)[source]

Get an evaluation run by ID.

Parameters:

run_id (str)

Return type:

GetRunResponse

async get_run_async(run_id)[source]

Get an evaluation run asynchronously.

Parameters:

run_id (str)

Return type:

GetRunResponse

list_runs(project=None, limit=100)[source]

List evaluation runs with optional filtering.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

GetRunsResponse

async list_runs_async(project=None, limit=100)[source]

List evaluation runs asynchronously.

Parameters:
  • project (str | None)

  • limit (int)

Return type:

GetRunsResponse

update_run(run_id, request)[source]

Update an evaluation run using UpdateRunRequest model.

Parameters:
Return type:

UpdateRunResponse

update_run_from_dict(run_id, run_data)[source]

Update an evaluation run from dictionary (legacy method).

Parameters:
Return type:

UpdateRunResponse

async update_run_async(run_id, request)[source]

Update an evaluation run asynchronously using UpdateRunRequest model.

Parameters:
Return type:

UpdateRunResponse

async update_run_from_dict_async(run_id, run_data)[source]

Update an evaluation run asynchronously from dictionary (legacy method).

Parameters:
Return type:

UpdateRunResponse

delete_run(run_id)[source]

Delete an evaluation run by ID.

Parameters:

run_id (str)

Return type:

DeleteRunResponse

async delete_run_async(run_id)[source]

Delete an evaluation run by ID asynchronously.

Parameters:

run_id (str)

Return type:

DeleteRunResponse

get_run_result(run_id, aggregate_function='average')[source]

Get aggregated result for a run from backend.

Backend Endpoint: GET /runs/:run_id/result?aggregate_function=<function>

The backend computes all aggregations, pass/fail status, and composite metrics.

Parameters:
  • run_id (str) – Experiment run ID

  • aggregate_function (str) – Aggregation function (“average”, “sum”, “min”, “max”)

Returns:

Dictionary with aggregated results from backend

Return type:

Dict[str, Any]

Example

>>> results = client.evaluations.get_run_result("run-123", "average")
>>> results["success"]
True
>>> results["metrics"]["accuracy"]
{'aggregate': 0.85, 'values': [0.8, 0.9, 0.85]}
async get_run_result_async(run_id, aggregate_function='average')[source]

Get aggregated result for a run asynchronously.

Parameters:
  • run_id (str)

  • aggregate_function (str)

Return type:

Dict[str, Any]

get_run_metrics(run_id)[source]

Get raw metrics for a run (without aggregation).

Backend Endpoint: GET /runs/:run_id/metrics

Parameters:

run_id (str) – Experiment run ID

Returns:

Dictionary with raw metrics data

Return type:

Dict[str, Any]

Example

>>> metrics = client.evaluations.get_run_metrics("run-123")
>>> metrics["events"]
[{'event_id': '...', 'metrics': {...}}, ...]
async get_run_metrics_async(run_id)[source]

Get raw metrics for a run asynchronously.

Parameters:

run_id (str)

Return type:

Dict[str, Any]

compare_runs(new_run_id, old_run_id, aggregate_function='average')[source]

Compare two experiment runs using backend aggregated comparison.

Backend Endpoint: GET /runs/:new_run_id/compare-with/:old_run_id

The backend computes metric deltas, percent changes, and datapoint differences.

Parameters:
  • new_run_id (str) – New experiment run ID

  • old_run_id (str) – Old experiment run ID

  • aggregate_function (str) – Aggregation function (“average”, “sum”, “min”, “max”)

Returns:

Dictionary with aggregated comparison data

Return type:

Dict[str, Any]

Example

>>> comparison = client.evaluations.compare_runs("run-new", "run-old")
>>> comparison["metric_deltas"]["accuracy"]
{'new_value': 0.85, 'old_value': 0.80, 'delta': 0.05}
async compare_runs_async(new_run_id, old_run_id, aggregate_function='average')[source]

Compare two experiment runs asynchronously (aggregated).

Parameters:
  • new_run_id (str)

  • old_run_id (str)

  • aggregate_function (str)

Return type:

Dict[str, Any]

compare_run_events(new_run_id, old_run_id, *, event_name=None, event_type=None, limit=100, page=1)[source]

Compare events between two experiment runs with datapoint-level matching.

Backend Endpoint: GET /runs/compare/events

The backend matches events by datapoint_id and provides detailed per-datapoint comparison with improved/degraded/same classification.

Parameters:
  • new_run_id (str) – New experiment run ID (run_id_1)

  • old_run_id (str) – Old experiment run ID (run_id_2)

  • event_name (str | None) – Optional event name filter (e.g., “initialization”)

  • event_type (str | None) – Optional event type filter (e.g., “session”)

  • limit (int) – Pagination limit (default: 100)

  • page (int) – Pagination page (default: 1)

Returns:

  • commonDatapoints: List of common datapoint IDs

  • metrics: Per-metric comparison with improved/degraded/same lists

  • events: Paired events (event_1, event_2) for each datapoint

  • event_details: Event presence information

  • old_run: Old run metadata

  • new_run: New run metadata

Return type:

Dictionary with detailed comparison including

Example

>>> comparison = client.evaluations.compare_run_events(
...     "run-new", "run-old",
...     event_name="initialization",
...     event_type="session"
... )
>>> len(comparison["commonDatapoints"])
3
>>> comparison["metrics"][0]["improved"]
["EXT-c1aed4cf0dfc3f16"]
async compare_run_events_async(new_run_id, old_run_id, *, event_name=None, event_type=None, limit=100, page=1)[source]

Compare events between two experiment runs asynchronously.

Parameters:
  • new_run_id (str)

  • old_run_id (str)

  • event_name (str | None)

  • event_type (str | None)

  • limit (int)

  • page (int)

Return type:

Dict[str, Any]

Example

from honeyhive import HoneyHive as Client

client = honeyhive.HoneyHive(api_key="your-api-key")

# Run evaluation
result = client.evaluations.evaluate(
    project="your-project",
    inputs={"query": "What is AI?"},
    ground_truth="Artificial Intelligence is...",
    evaluators=["exact_match", "semantic_similarity"]
)

EventsAPI

API client for event operations.

class honeyhive.api.events.EventsAPI(client)[source]

Bases: BaseAPI

API for event operations.

Parameters:

client (HoneyHive)

create_event(event)[source]

Create a new event using CreateEventRequest model.

Parameters:

event (CreateEventRequest)

Return type:

CreateEventResponse

create_event_from_dict(event_data)[source]

Create a new event from event data dictionary (legacy method).

Parameters:

event_data (dict)

Return type:

CreateEventResponse

create_event_from_request(event)[source]

Create a new event from CreateEventRequest object.

Parameters:

event (CreateEventRequest)

Return type:

CreateEventResponse

async create_event_async(event)[source]

Create a new event asynchronously using CreateEventRequest model.

Parameters:

event (CreateEventRequest)

Return type:

CreateEventResponse

async create_event_from_dict_async(event_data)[source]

Create a new event asynchronously from event data dictionary (legacy method).

Parameters:

event_data (dict)

Return type:

CreateEventResponse

async create_event_from_request_async(event)[source]

Create a new event asynchronously.

Parameters:

event (CreateEventRequest)

Return type:

CreateEventResponse

delete_event(event_id)[source]

Delete an event by ID.

Parameters:

event_id (str)

Return type:

bool

async delete_event_async(event_id)[source]

Delete an event by ID asynchronously.

Parameters:

event_id (str)

Return type:

bool

update_event(request)[source]

Update an event.

Parameters:

request (UpdateEventRequest)

Return type:

None

async update_event_async(request)[source]

Update an event asynchronously.

Parameters:

request (UpdateEventRequest)

Return type:

None

create_event_batch(request)[source]

Create multiple events using BatchCreateEventRequest model.

Parameters:

request (BatchCreateEventRequest)

Return type:

BatchCreateEventResponse

create_event_batch_from_list(events)[source]

Create multiple events from a list of CreateEventRequest objects.

Parameters:

events (List[CreateEventRequest])

Return type:

BatchCreateEventResponse

async create_event_batch_async(request)[source]

Create multiple events asynchronously using BatchCreateEventRequest model.

Parameters:

request (BatchCreateEventRequest)

Return type:

BatchCreateEventResponse

async create_event_batch_from_list_async(events)[source]

Create multiple events asynchronously from a list of CreateEventRequest objects.

Parameters:

events (List[CreateEventRequest])

Return type:

BatchCreateEventResponse

list_events(event_filters, limit=100, project=None, page=1)[source]

List events using EventFilter model with dynamic processing optimization.

Uses the proper /events/export POST endpoint as specified in OpenAPI spec.

Parameters:
  • event_filters (EventFilter | List[EventFilter]) – EventFilter or list of EventFilter objects with filtering criteria

  • limit (int) – Maximum number of events to return (default: 100)

  • project (str | None) – Project name to filter by (required by API)

  • page (int) – Page number for pagination (default: 1)

Returns:

List of Event objects matching the filters

Return type:

List[Event]

Examples

Filter events by type and status:

filters = [
    EventFilter(
        field="event_type",
        operator="is",
        value="model",
        type="string",
    ),
    EventFilter(
        field="error",
        operator="is not",
        value=None,
        type="string",
    ),
]
events = client.events.list_events(
    event_filters=filters,
    project="My Project",
    limit=50
)
list_events_from_dict(event_filter, limit=100)[source]

List events from filter dictionary (legacy method).

Parameters:
Return type:

List[Event]

get_events(project, filters, *, date_range=None, limit=1000, page=1)[source]

Get events using filters via /events/export endpoint.

This is the proper way to filter events by session_id and other criteria.

Parameters:
  • project (str) – Name of the project associated with the event

  • filters (List[EventFilter]) – List of EventFilter objects to apply

  • date_range (Dict[str, str] | None) – Optional date range filter with $gte and $lte ISO strings

  • limit (int) – Limit number of results (default 1000, max 7500)

  • page (int) – Page number of results (default 1)

Returns:

Dict containing ‘events’ list and ‘totalEvents’ count

Return type:

Dict[str, Any]

async list_events_async(event_filters, limit=100, project=None, page=1)[source]

List events asynchronously using EventFilter model.

Uses the proper /events/export POST endpoint as specified in OpenAPI spec.

Parameters:
  • event_filters (EventFilter | List[EventFilter]) – EventFilter or list of EventFilter objects with filtering criteria

  • limit (int) – Maximum number of events to return (default: 100)

  • project (str | None) – Project name to filter by (required by API)

  • page (int) – Page number for pagination (default: 1)

Returns:

List of Event objects matching the filters

Return type:

List[Event]

Examples

Filter events by type and status:

filters = [
    EventFilter(
        field="event_type",
        operator="is",
        value="model",
        type="string",
    ),
    EventFilter(
        field="error",
        operator="is not",
        value=None,
        type="string",
    ),
]
events = await client.events.list_events_async(
    event_filters=filters,
    project="My Project",
    limit=50
)
async list_events_from_dict_async(event_filter, limit=100)[source]

List events asynchronously from filter dictionary (legacy method).

Parameters:
Return type:

List[Event]

Example

from honeyhive import HoneyHive as Client

client = honeyhive.HoneyHive(api_key="your-api-key")

# Send event
client.events.send_event(
    project="your-project",
    event_type="llm_call",
    event_data={
        "model": "gpt-4",
        "input": "Hello",
        "output": "Hi there!",
        "latency": 250
    }
)

See Also