honeyhive.tracer.lifecycle

Tracer lifecycle management for shutdown and cleanup operations.

This module provides comprehensive lifecycle management for HoneyHive tracer instances, including graceful shutdown, resource cleanup, and proper handling of multi-instance architectures.

The module is organized into sub-components but maintains a unified public API for backward compatibility with existing usage patterns.

is_shutdown_detected

is_shutdown_detected() -> bool

Check if shutdown has been detected (for internal use by tracer components).

This function dynamically detects shutdown conditions and is safe to call from any tracer component that needs to check shutdown state.

Returns:

Type	Description
bool	True if shutdown is in progress, False otherwise

Source code in src/honeyhive/utils/logger.py

def is_shutdown_detected() -> bool:
    """Check if shutdown has been detected (for internal use by tracer components).

    This function dynamically detects shutdown conditions and is safe to call
    from any tracer component that needs to check shutdown state.

    Returns:
        True if shutdown is in progress, False otherwise
    """
    return _detect_shutdown_conditions()

disable_new_span_creation

disable_new_span_creation() -> None

Disable creation of new spans during shutdown phase.

This function prevents new spans from being created while allowing existing spans to complete naturally. This is part of the graceful shutdown process to prevent data loss.

Usage:

.. code-block:: python

# Phase 1: Graceful drain
disable_new_span_creation()
time.sleep(0.1)  # Allow existing spans to complete

# Phase 2: Force flush
tracer.force_flush()

Source code in src/honeyhive/tracer/lifecycle/core.py

def disable_new_span_creation() -> None:
    """Disable creation of new spans during shutdown phase.

    This function prevents new spans from being created while allowing
    existing spans to complete naturally. This is part of the graceful
    shutdown process to prevent data loss.

    **Usage:**

    .. code-block:: python

        # Phase 1: Graceful drain
        disable_new_span_creation()
        time.sleep(0.1)  # Allow existing spans to complete

        # Phase 2: Force flush
        tracer.force_flush()
    """
    _new_spans_disabled.set()

is_new_span_creation_disabled

is_new_span_creation_disabled() -> bool

Check if new span creation is disabled.

Returns:

Name	Type	Description
bool	bool	True if new span creation is disabled, False otherwise

Source code in src/honeyhive/tracer/lifecycle/core.py

def is_new_span_creation_disabled() -> bool:
    """Check if new span creation is disabled.

    Returns:
        bool: True if new span creation is disabled, False otherwise
    """
    return _new_spans_disabled.is_set()

register_tracer_for_atexit_cleanup

register_tracer_for_atexit_cleanup(
    tracer_instance: Any,
) -> None

Register a tracer instance for automatic cleanup on Python exit.

This function provides thread-safe registration of tracer instances for automatic cleanup during Python interpreter shutdown. This prevents race conditions in pytest-xdist workers where logging streams are closed before fixture teardown runs.

:param tracer_instance: The tracer instance to register for cleanup :type tracer_instance: HoneyHiveTracer

Thread Safety:

This function is thread-safe and supports the multi-instance architecture. Multiple tracer instances can be registered concurrently from different threads without conflicts.

Example:

.. code-block:: python

# Register tracer for automatic cleanup
tracer = HoneyHiveTracer.init(api_key="...", project="...")
register_tracer_for_atexit_cleanup(tracer)

# Tracer will be automatically cleaned up on Python exit
# even if pytest-xdist closes logging streams early

Note:

Uses WeakSet to avoid keeping tracers alive just for cleanup. If a tracer is garbage collected, it's automatically removed from the cleanup registry.

Source code in src/honeyhive/tracer/lifecycle/core.py

def register_tracer_for_atexit_cleanup(tracer_instance: Any) -> None:
    """Register a tracer instance for automatic cleanup on Python exit.

    This function provides thread-safe registration of tracer instances
    for automatic cleanup during Python interpreter shutdown. This prevents
    race conditions in pytest-xdist workers where logging streams are closed
    before fixture teardown runs.

    :param tracer_instance: The tracer instance to register for cleanup
    :type tracer_instance: HoneyHiveTracer

    **Thread Safety:**

    This function is thread-safe and supports the multi-instance architecture.
    Multiple tracer instances can be registered concurrently from different
    threads without conflicts.

    **Example:**

    .. code-block:: python

        # Register tracer for automatic cleanup
        tracer = HoneyHiveTracer.init(api_key="...", project="...")
        register_tracer_for_atexit_cleanup(tracer)

        # Tracer will be automatically cleaned up on Python exit
        # even if pytest-xdist closes logging streams early

    **Note:**

    Uses WeakSet to avoid keeping tracers alive just for cleanup.
    If a tracer is garbage collected, it's automatically removed
    from the cleanup registry.
    """
    with _lifecycle_lock:
        # Check if already registered to prevent duplicate atexit handlers
        if tracer_instance in _registered_tracers:
            safe_log(
                tracer_instance,
                "debug",
                f"Tracer already registered for atexit cleanup: {id(tracer_instance)}",
            )
            return

        # Add to registry using WeakSet (won't keep tracer alive)
        _registered_tracers.add(tracer_instance)

        # Create cleanup function that captures tracer by weak reference
        tracer_ref = weakref.ref(tracer_instance)

        def cleanup_tracer_on_exit() -> None:
            """Cleanup function that runs during Python shutdown."""
            # safe_log will automatically detect shutdown conditions

            tracer = tracer_ref()
            if tracer is not None:
                try:
                    # Import here to avoid circular imports
                    # pylint: disable=import-outside-toplevel
                    from .flush import force_flush_tracer
                    from .shutdown import shutdown_tracer

                    # Force flush first, then shutdown (no logging during shutdown)
                    force_flush_tracer(tracer, timeout_millis=1000)  # Shorter timeout
                    shutdown_tracer(tracer)

                except Exception as e:
                    # Graceful degradation
                    # Silent failure during shutdown is expected, but log for debugging
                    safe_log(
                        None,
                        "debug",
                        "Expected shutdown exception during cleanup",
                        honeyhive_data={"error_type": type(e).__name__},
                    )

        # Register the cleanup function with atexit
        atexit.register(cleanup_tracer_on_exit)

        safe_log(
            tracer_instance,
            "debug",
            f"Registered tracer for atexit cleanup: {id(tracer_instance)}",
            honeyhive_data={
                "tracer_id": id(tracer_instance),
                "registered_count": len(_registered_tracers),
                "architecture": "multi-instance",
            },
        )

unregister_tracer_from_atexit_cleanup

unregister_tracer_from_atexit_cleanup(
    tracer_instance: Any,
) -> None

Unregister a tracer instance from automatic cleanup.

This function removes a tracer from the atexit cleanup registry. Useful when manually shutting down a tracer before Python exit.

:param tracer_instance: The tracer instance to unregister :type tracer_instance: HoneyHiveTracer

Thread Safety:

This function is thread-safe and supports concurrent access.

Example:

.. code-block:: python

# Manual cleanup - unregister from atexit
unregister_tracer_from_atexit_cleanup(tracer)
tracer.force_flush()
tracer.shutdown()

Source code in src/honeyhive/tracer/lifecycle/core.py

def unregister_tracer_from_atexit_cleanup(tracer_instance: Any) -> None:
    """Unregister a tracer instance from automatic cleanup.

    This function removes a tracer from the atexit cleanup registry.
    Useful when manually shutting down a tracer before Python exit.

    :param tracer_instance: The tracer instance to unregister
    :type tracer_instance: HoneyHiveTracer

    **Thread Safety:**

    This function is thread-safe and supports concurrent access.

    **Example:**

    .. code-block:: python

        # Manual cleanup - unregister from atexit
        unregister_tracer_from_atexit_cleanup(tracer)
        tracer.force_flush()
        tracer.shutdown()
    """
    with _lifecycle_lock:
        if tracer_instance in _registered_tracers:
            _registered_tracers.discard(tracer_instance)
            safe_log(
                tracer_instance,
                "debug",
                f"Unregistered tracer from atexit cleanup: {id(tracer_instance)}",
                honeyhive_data={
                    "tracer_id": id(tracer_instance),
                    "remaining_count": len(_registered_tracers),
                },
            )

force_flush_tracer

force_flush_tracer(
    tracer_instance: Any, timeout_millis: float = 30000
) -> bool

Force flush any pending spans and data for a tracer instance.

This function ensures that all pending spans and telemetry data are immediately sent to their destinations, rather than waiting for automatic batching/flushing. It handles multiple flush targets and provides comprehensive error handling.

:param tracer_instance: The tracer instance to flush :type tracer_instance: HoneyHiveTracer :param timeout_millis: Maximum time to wait for flush completion in milliseconds :type timeout_millis: float :return: True if flush completed successfully within timeout, False otherwise :rtype: bool

Example:

.. code-block:: python

# Flush with default timeout (30 seconds)
success = force_flush_tracer(tracer)

# Flush with custom timeout (5 seconds)
success = force_flush_tracer(tracer, timeout_millis=5000)

# Use before critical sections
if force_flush_tracer(tracer):
    print("All spans flushed successfully")
else:
    print("Flush timeout or error occurred")

Note:

This function attempts to flush multiple components in sequence: the tracer provider, custom span processors, and batch processors. It returns True only if all components flush successfully.

Source code in src/honeyhive/tracer/lifecycle/flush.py

def force_flush_tracer(tracer_instance: Any, timeout_millis: float = 30000) -> bool:
    """Force flush any pending spans and data for a tracer instance.

    This function ensures that all pending spans and telemetry data are
    immediately sent to their destinations, rather than waiting for
    automatic batching/flushing. It handles multiple flush targets and
    provides comprehensive error handling.

    :param tracer_instance: The tracer instance to flush
    :type tracer_instance: HoneyHiveTracer
    :param timeout_millis: Maximum time to wait for flush completion in milliseconds
    :type timeout_millis: float
    :return: True if flush completed successfully within timeout, False otherwise
    :rtype: bool

    **Example:**

    .. code-block:: python

        # Flush with default timeout (30 seconds)
        success = force_flush_tracer(tracer)

        # Flush with custom timeout (5 seconds)
        success = force_flush_tracer(tracer, timeout_millis=5000)

        # Use before critical sections
        if force_flush_tracer(tracer):
            print("All spans flushed successfully")
        else:
            print("Flush timeout or error occurred")

    **Note:**

    This function attempts to flush multiple components in sequence:
    the tracer provider, custom span processors, and batch processors.
    It returns True only if all components flush successfully.
    """
    safe_log(tracer_instance, "debug", "Force flush requested")

    flush_results: List[Tuple[str, bool]] = []

    safe_log(
        tracer_instance,
        "debug",
        f"force_flush_tracer: Acquiring _lifecycle_lock (timeout: {timeout_millis}ms)",
    )

    try:
        safe_log(
            tracer_instance,
            "debug",
            "force_flush_tracer: Attempting to acquire _lifecycle_lock...",
        )
        # Use environment-optimized flush timeout
        flush_timeout_seconds = timeout_millis / 1000.0
        with acquire_lifecycle_lock_optimized(
            "flush", custom_timeout=flush_timeout_seconds
        ) as acquired:
            if not acquired:
                safe_log(
                    tracer_instance,
                    "warning",
                    f"Failed to acquire _lifecycle_lock ({flush_timeout_seconds}s)",
                )
                return False
            safe_log(
                tracer_instance,
                "debug",
                "force_flush_tracer: Successfully acquired _lifecycle_lock",
            )
            # 1. Flush the tracer provider if available and supports it
            _flush_tracer_provider(tracer_instance, timeout_millis, flush_results)

            # 2. Flush our custom span processor if available
            _flush_span_processor(tracer_instance, timeout_millis, flush_results)

            # 3. Flush any batch span processors attached to the provider
            _flush_batch_processors(tracer_instance, timeout_millis, flush_results)

        # Calculate overall result
        overall_success = all(result for _, result in flush_results)

        _log_flush_results(tracer_instance, overall_success, flush_results)

        return overall_success

    except Exception as e:
        # Graceful degradation - never crash host
        safe_log(
            tracer_instance,
            "error",
            "Force flush failed",
            honeyhive_data={
                "error": str(e),
                "error_type": type(e).__name__,
                "operation": "force_flush_tracer",
            },
        )
        return False

graceful_shutdown_all

graceful_shutdown_all() -> None

Gracefully shutdown all registered tracer instances.

This function attempts to find and shutdown all active HoneyHive tracer instances. It's useful for application shutdown or cleanup scenarios where multiple tracers might be active.

Example:

.. code-block:: python

# Application shutdown
import atexit
atexit.register(graceful_shutdown_all)

# Or explicit cleanup
graceful_shutdown_all()

Note:

This function uses the tracer registry to find active instances. It attempts graceful shutdown but continues even if some instances fail to shutdown properly.

Source code in src/honeyhive/tracer/lifecycle/shutdown.py

def graceful_shutdown_all() -> None:
    """Gracefully shutdown all registered tracer instances.

    This function attempts to find and shutdown all active HoneyHive
    tracer instances. It's useful for application shutdown or cleanup
    scenarios where multiple tracers might be active.

    **Example:**

    .. code-block:: python

        # Application shutdown
        import atexit
        atexit.register(graceful_shutdown_all)

        # Or explicit cleanup
        graceful_shutdown_all()

    **Note:**

    This function uses the tracer registry to find active instances.
    It attempts graceful shutdown but continues even if some instances
    fail to shutdown properly.
    """
    try:
        active_tracers = registry.get_all_tracers()

        if not active_tracers:
            safe_log(None, "debug", "No active tracers found for shutdown")
            return

        safe_log(
            None,
            "info",
            "Starting graceful shutdown of all tracers",
            honeyhive_data={"tracer_count": len(active_tracers)},
        )

        shutdown_results = []

        for tracer_instance in active_tracers:
            try:
                shutdown_tracer(tracer_instance)
                shutdown_results.append(
                    (getattr(tracer_instance, "_tracer_id", "unknown"), True)
                )
                safe_log(
                    tracer_instance,
                    "debug",
                    "Tracer shutdown successful",
                    honeyhive_data={
                        "tracer_id": getattr(tracer_instance, "_tracer_id", "unknown")
                    },
                )
            except Exception as e:
                shutdown_results.append(
                    (getattr(tracer_instance, "_tracer_id", "unknown"), False)
                )
                # Graceful degradation - never crash host
                safe_log(
                    tracer_instance,
                    "error",
                    "Tracer shutdown failed",
                    honeyhive_data={
                        "tracer_id": getattr(tracer_instance, "_tracer_id", "unknown"),
                        "error": str(e),
                        "error_type": type(e).__name__,
                        "operation": "graceful_shutdown_single_tracer",
                    },
                )

        # Log summary
        successful_shutdowns = sum(1 for _, success in shutdown_results if success)
        safe_log(
            None,
            "info",
            "Graceful shutdown completed",
            honeyhive_data={
                "total_tracers": len(active_tracers),
                "successful_shutdowns": successful_shutdowns,
                "failed_shutdowns": len(active_tracers) - successful_shutdowns,
            },
        )

    except Exception as e:
        # Graceful degradation - never crash host
        safe_log(
            None,
            "error",
            "Error during graceful shutdown of all tracers",
            honeyhive_data={
                "error": str(e),
                "error_type": type(e).__name__,
                "operation": "graceful_shutdown_all",
            },
        )

shutdown_tracer

shutdown_tracer(tracer_instance: Any) -> None

Shutdown a tracer instance and clean up its resources.

This function performs a graceful shutdown of a tracer instance, including flushing pending data, shutting down providers, and cleaning up resources. It handles both main and secondary providers appropriately.

:param tracer_instance: The tracer instance to shutdown :type tracer_instance: HoneyHiveTracer

Example:

.. code-block:: python

# Graceful shutdown
shutdown_tracer(tracer)

# In a try/finally block
try:
    # Use tracer
    with tracer.start_span("operation") as span:
        pass
finally:
    shutdown_tracer(tracer)

Note:

This function only shuts down the OpenTelemetry provider if the tracer instance is the main provider. Secondary providers are left running to avoid disrupting other tracer instances.

Source code in src/honeyhive/tracer/lifecycle/shutdown.py

def shutdown_tracer(tracer_instance: Any) -> None:
    """Shutdown a tracer instance and clean up its resources.

    This function performs a graceful shutdown of a tracer instance,
    including flushing pending data, shutting down providers, and
    cleaning up resources. It handles both main and secondary providers
    appropriately.

    :param tracer_instance: The tracer instance to shutdown
    :type tracer_instance: HoneyHiveTracer

    **Example:**

    .. code-block:: python

        # Graceful shutdown
        shutdown_tracer(tracer)

        # In a try/finally block
        try:
            # Use tracer
            with tracer.start_span("operation") as span:
                pass
        finally:
            shutdown_tracer(tracer)

    **Note:**

    This function only shuts down the OpenTelemetry provider if the
    tracer instance is the main provider. Secondary providers are
    left running to avoid disrupting other tracer instances.
    """
    # Check if logging is still available (pytest-xdist workers may have closed streams)
    safe_log(
        tracer_instance, "debug", "shutdown_tracer: Starting data loss prevention phase"
    )

    # Phase 1: Data loss prevention - optimized for parallel execution
    # This ensures we attempt to preserve data even if locking fails
    test_mode = getattr(tracer_instance, "test_mode", False)

    # Skip data loss prevention in test mode to prevent worker conflicts
    # In production, this is critical for data preservation
    if not test_mode:
        # Graceful drain phase (production only)
        # For multi-instance architecture: only disable globally if main provider
        if getattr(tracer_instance, "is_main_provider", False):
            disable_new_span_creation()

        # Always set instance-specific shutdown flag for this tracer
        # Protected access required for multi-instance lifecycle management
        tracer_instance._instance_shutdown = True  # pylint: disable=protected-access

        # Brief grace period for existing spans to complete naturally
        time.sleep(0.1)

        # Force flush with extended timeout and retry logic (before lock acquisition)
        timeout_ms = 5000  # Extended timeout for production

        safe_log(
            tracer_instance,
            "debug",
            "Starting pre-lock force flush for data loss prevention",
            honeyhive_data={
                "timeout_ms": timeout_ms,
                "test_mode": test_mode,
                "phase": "pre_lock_data_preservation",
            },
        )

        flush_success = force_flush_tracer(tracer_instance, timeout_millis=timeout_ms)

        # Retry logic for critical data preservation (production only)
        if not flush_success:
            safe_log(
                tracer_instance,
                "warning",
                f"Pre-lock flush failed (timeout: {timeout_ms}ms), retrying",
            )

            retry_timeout_ms = timeout_ms * 2
            flush_success = force_flush_tracer(
                tracer_instance, timeout_millis=retry_timeout_ms
            )

            if flush_success:
                safe_log(
                    tracer_instance,
                    "info",
                    f"Pre-lock flush succeeded on retry ({retry_timeout_ms}ms)",
                )
            else:
                safe_log(
                    tracer_instance,
                    "error",
                    f"Pre-lock flush failed after retry ({retry_timeout_ms}ms), "
                    "continuing with shutdown - potential data loss",
                )
    else:
        # Test mode: skip pre-lock flush to prevent pytest-xdist worker conflicts
        safe_log(
            tracer_instance,
            "debug",
            "Skipping pre-lock flush in test mode to prevent conflicts",
        )
        flush_success = True  # Assume success for test mode

    safe_log(
        tracer_instance,
        "debug",
        "shutdown_tracer: Acquiring _lifecycle_lock for shutdown",
    )

    # Use environment-optimized lock timeout for better performance
    # Automatically detects Lambda, K8s, high-concurrency environments
    with acquire_lifecycle_lock_optimized("lifecycle") as lock_acquired:
        if not lock_acquired:
            # Graceful degradation: Try to log timeout but don't crash
            config = get_lock_config()
            timeout_used = config.get("lifecycle_timeout", 1.0)
            safe_log(
                tracer_instance,
                "warning",
                f"Failed to acquire _lifecycle_lock within {timeout_used}s, "
                "proceeding without lock",
                honeyhive_data={
                    "lock_timeout": timeout_used,
                    "lock_strategy": config.get("description", "unknown"),
                    "degradation_reason": "lock_acquisition_timeout",
                    "data_flush_completed": flush_success,
                },
            )
            # Continue without the lock - better than hanging indefinitely
            _shutdown_without_lock(tracer_instance)
            return

        try:
            safe_log(
                tracer_instance,
                "debug",
                "Starting tracer shutdown",
                honeyhive_data={
                    "is_main_provider": tracer_instance.is_main_provider,
                    "has_provider": bool(tracer_instance.provider),
                },
            )

            # Skip force_flush during shutdown to prevent recursive deadlock
            # The force_flush_tracer also tries to acquire _lifecycle_lock,
            # causing deadlock
            safe_log(
                tracer_instance,
                "debug",
                "Skipping force_flush during shutdown to prevent recursive deadlock",
            )

            # Only shutdown if we're the main provider
            if (
                tracer_instance.is_main_provider
                and tracer_instance.provider
                and hasattr(tracer_instance.provider, "shutdown")
            ):
                _shutdown_main_provider(tracer_instance)
            else:
                _cleanup_secondary_provider(tracer_instance)

            # Clean up instance state
            _cleanup_tracer_state(tracer_instance)

        except Exception as e:
            # Graceful degradation - never crash host
            safe_log(
                tracer_instance,
                "error",
                "Error during tracer shutdown",
                honeyhive_data={
                    "error": str(e),
                    "error_type": type(e).__name__,
                    "operation": "tracer_shutdown",
                },
            )

wait_for_pending_spans

wait_for_pending_spans(
    tracer_instance: Any, max_wait_seconds: float = 10.0
) -> bool

Wait for pending spans to complete processing.

This function waits for any pending spans in the tracer's processors to complete processing. It's useful before shutdown to ensure all data is properly sent.

:param tracer_instance: The tracer instance to wait for :type tracer_instance: HoneyHiveTracer :param max_wait_seconds: Maximum time to wait in seconds :type max_wait_seconds: float :return: True if all spans completed within timeout, False otherwise :rtype: bool

Example:

.. code-block:: python

# Wait for spans before shutdown
if wait_for_pending_spans(tracer, max_wait_seconds=5.0):
    print("All spans completed")
else:
    print("Timeout waiting for spans")

Note:

This function polls the span processors to check for pending work. It's a best-effort operation and may not catch all edge cases.

Source code in src/honeyhive/tracer/lifecycle/shutdown.py

def wait_for_pending_spans(
    tracer_instance: Any, max_wait_seconds: float = 10.0
) -> bool:
    """Wait for pending spans to complete processing.

    This function waits for any pending spans in the tracer's processors
    to complete processing. It's useful before shutdown to ensure all
    data is properly sent.

    :param tracer_instance: The tracer instance to wait for
    :type tracer_instance: HoneyHiveTracer
    :param max_wait_seconds: Maximum time to wait in seconds
    :type max_wait_seconds: float
    :return: True if all spans completed within timeout, False otherwise
    :rtype: bool

    **Example:**

    .. code-block:: python

        # Wait for spans before shutdown
        if wait_for_pending_spans(tracer, max_wait_seconds=5.0):
            print("All spans completed")
        else:
            print("Timeout waiting for spans")

    **Note:**

    This function polls the span processors to check for pending work.
    It's a best-effort operation and may not catch all edge cases.
    """
    if not tracer_instance.provider:
        return True

    start_time = time.time()

    while time.time() - start_time < max_wait_seconds:
        try:
            if not _has_pending_spans(tracer_instance):
                safe_log(
                    tracer_instance,
                    "debug",
                    "No pending spans detected",
                    honeyhive_data={"wait_time": time.time() - start_time},
                )
                return True

            # Wait a bit before checking again
            time.sleep(0.1)

        except Exception as e:
            # Graceful degradation - never crash host
            safe_log(
                tracer_instance,
                "warning",
                f"Error checking for pending spans: {e}",
                honeyhive_data={
                    "wait_time": time.time() - start_time,
                    "error_type": type(e).__name__,
                    "operation": "wait_for_pending_spans",
                },
            )
            break

    safe_log(
        tracer_instance,
        "warning",
        "Timeout waiting for pending spans",
        honeyhive_data={"max_wait_seconds": max_wait_seconds},
    )
    return False