Guides

Best open-source LLM observability tools

Buyer's guide

Best Open-Source LLM Observability Tools in 2026 (Self-Hosted)

Last updated · By the OpenLIT team

Short answer

The best open-source LLM observability tools in 2026 are OpenLIT (Apache 2.0), Langfuse (MIT core), Arize Phoenix (Elastic License 2.0), Comet Opik (Apache 2.0), MLflow (Apache 2.0), Helicone (Apache 2.0), and OpenLLMetry (Apache 2.0 instrumentation). OpenLIT is the most complete Apache-2.0, OpenTelemetry-native option, covering tracing, evals, guardrails, prompts, and GPU monitoring in one self-hosted platform.

Open-source LLM observability lets you keep prompts, completions, and traces on your own infrastructure, avoid per-trace pricing, and change vendors later. The license matters: Apache 2.0 and MIT are OSI-approved, while Elastic License 2.0 is source-available with restrictions.

This guide covers the open-source and source-available tools teams self-host today. We build OpenLIT and list it first; each tool has the same criteria and honest limitations.

The list at a glance

  1. OpenLIT: Open-source, OpenTelemetry-native agent and LLM observability in one self-hosted platform
  2. Langfuse: Popular open-source tracing, prompt management, and evals with a managed cloud
  3. Arize Phoenix: Notebook-friendly tracing and evals, with an upgrade path to Arize AX
  4. Comet Opik: Apache-2.0 tracing and evals from the Comet ML team
  5. MLflow: Teams already using MLflow or Databricks
  6. Helicone: Proxy-based request logging and cost tracking with minimal code
  7. OpenLLMetry (Traceloop): OpenTelemetry instrumentation you send to an existing backend

Comparison table

ToolLicenseSelf-hostOpenTelemetry
OpenLITApache 2.0Yes (Docker, Kubernetes Helm). Free.Native: every span and metric is standard OTLP
LangfuseMIT core + commercial enterprise featuresYesAccepts OTel traces (v3+); primary SDKs are Langfuse-specific
Arize PhoenixElastic License 2.0 (source-available)YesOpenTelemetry via OpenInference instrumentation
Comet OpikApache 2.0YesOTel-compatible options
MLflowApache 2.0Yes (managed on Databricks)MLflow Tracing is OpenTelemetry-compatible
HeliconeApache 2.0YesNot OTel-native (proxy / gateway based)
OpenLLMetry (Traceloop)Apache 2.0 (SDK)SDK only; Traceloop platform is commercialNative

1. OpenLIT

Best for: Open-source, OpenTelemetry-native agent and LLM observability in one self-hosted platform

OpenLIT is an open-source AI agent and LLM observability platform built on OpenTelemetry. It combines tracing, cost and token tracking, evals (LLM-as-a-judge), guardrails, Prompt Hub, a secrets Vault, model comparison (OpenGround), coding agent monitoring, and GPU monitoring in one Apache-2.0 platform you self-host on ClickHouse.

Strengths

  • Apache 2.0 with no per-trace fees; data stays in your own ClickHouse
  • OpenTelemetry-native SDKs for Python, JavaScript, and Go that auto-instrument LLM providers, agent frameworks, vector DBs, and MCP
  • Coding agent observability for Claude Code, Codex, and Cursor through the openlit CLI, no SDK needed
  • GPU monitoring for NVIDIA, AMD, and Intel next to inference traces
  • Exports OTLP to Grafana, Datadog, or any OpenTelemetry backend, so there is no lock-in

Limitations

  • Managed cloud is not generally available yet; today you self-host
  • Smaller community than Langfuse or LangSmith
  • RBAC, audit logs, alerts, and the eBPF zero-code controller are in the paid Enterprise Edition

License: Apache 2.0 · Self-host: Yes (Docker, Kubernetes Helm). Free. · Website

2. Langfuse

Best for: Popular open-source tracing, prompt management, and evals with a managed cloud

Langfuse is one of the most widely adopted open-source LLM engineering platforms, covering tracing, prompt management, datasets, and evaluations, with both self-hosted and cloud options.

Strengths

  • Large community and many framework integrations
  • Mature prompt management, datasets, and annotation workflows
  • Managed cloud available

Limitations

  • Some enterprise features are under a commercial license
  • No GPU or infrastructure monitoring

License: MIT core + commercial enterprise features · Self-host: Yes · Website · OpenLIT vs Langfuse

3. Arize Phoenix

Best for: Notebook-friendly tracing and evals, with an upgrade path to Arize AX

Arize Phoenix is an AI observability and evaluation tool from Arize, built on OpenTelemetry and OpenInference. Arize AX is the commercial enterprise platform.

Strengths

  • Strong evaluation and experiment tooling
  • Good fit for data science and notebook workflows
  • Clear path to the enterprise Arize AX platform

Limitations

  • Elastic License 2.0 is not OSI-approved open source, which matters to some legal teams
  • No built-in GPU monitoring or secrets management

License: Elastic License 2.0 (source-available) · Self-host: Yes · Website · OpenLIT vs Arize Phoenix

4. Comet Opik

Best for: Apache-2.0 tracing and evals from the Comet ML team

Opik is an open-source LLM evaluation and tracing platform from Comet, with self-hosted and cloud deployments.

Strengths

  • Apache 2.0 license
  • Evaluation-focused feature set with experiments and datasets
  • Fits teams already using Comet for ML experiment tracking

Limitations

  • Limited infrastructure and GPU monitoring
  • Less focus on OTLP export to other backends

License: Apache 2.0 · Self-host: Yes · Website · OpenLIT vs Comet Opik

5. MLflow

Best for: Teams already using MLflow or Databricks

MLflow is the open-source ML lifecycle platform. MLflow Tracing and its GenAI evaluation features add LLM and agent observability to the same tool used for experiments and model registry.

Strengths

  • Apache 2.0 and very widely deployed
  • One tool for classic ML and GenAI experiments
  • Managed option on Databricks

Limitations

  • LLM observability is one part of a broader ML platform
  • No GPU monitoring, guardrails, or secrets management for LLM apps

License: Apache 2.0 · Self-host: Yes (managed on Databricks) · Website

6. Helicone

Best for: Proxy-based request logging and cost tracking with minimal code

Helicone is an open-source LLM observability tool and AI gateway that logs requests by sitting as a proxy between your app and the model provider.

Strengths

  • One-line setup by changing the provider base URL
  • Request logging, caching, and cost tracking at the gateway

Limitations

  • Proxy sits in the request path
  • Limited visibility into multi-step agent internals, tool calls, and vector DBs

License: Apache 2.0 · Self-host: Yes · Website · OpenLIT vs Helicone

7. OpenLLMetry (Traceloop)

Best for: OpenTelemetry instrumentation you send to an existing backend

OpenLLMetry is an Apache-2.0 set of OpenTelemetry instrumentations for LLM providers and frameworks, maintained by Traceloop. It is an instrumentation library, not a full platform.

Strengths

  • OpenTelemetry-native instrumentation
  • Works with any OTLP backend

Limitations

  • No built-in UI, evals, prompt management, or storage; you need a backend
  • Traceloop platform features are commercial

License: Apache 2.0 (SDK) · Self-host: SDK only; Traceloop platform is commercial · Website · OpenLIT vs OpenLLMetry (Traceloop)

How to choose

If you needPick
Strict OSI-approved license (Apache 2.0)OpenLIT, Opik, or MLflow
Largest open-source communityLangfuse
OpenTelemetry-native end to endOpenLIT
GPU monitoring for self-hosted inferenceOpenLIT
Instrumentation only, existing OTel backendOpenLLMetry or the OpenLIT SDK

Evaluation criteria

Agent trace depth
Does it capture the full trajectory (LLM calls, tool calls, MCP requests, retrieval, and sub-agents) or only individual model requests?
OpenTelemetry support
Native OTLP output keeps your instrumentation portable. Proprietary SDKs make switching tools expensive later.
Evaluations
LLM-as-a-judge, programmatic checks, and human review on production traces, plus offline evals you can run in CI.
Cost and token tracking
Per request, per user, per model, and per agent run, with custom model pricing.
License and self-hosting
OSI-approved open source (Apache 2.0, MIT) versus source-available or closed. Matters for data residency and regulated teams.
Prompt management and guardrails
Versioned prompts you can roll back without redeploying, and runtime checks for prompt injection, PII, and unsafe output.

Methodology: we compared public documentation, licenses, and repositories for each tool as of November 18, 2026. OpenLIT is our product. Pricing changes often, so we do not list prices; check each vendor's site. Spot an error? Email [email protected].

FAQ

What is the best open-source LLM observability tool?

OpenLIT is the most complete Apache-2.0, OpenTelemetry-native option, with tracing, evals, guardrails, Prompt Hub, Vault, coding agent monitoring, and GPU monitoring. Langfuse is the most popular MIT-licensed option with a managed cloud.

Is Arize Phoenix open source?

Phoenix is source-available under the Elastic License 2.0, which is not an OSI-approved open-source license. OpenLIT, Opik, and MLflow are Apache 2.0.

Can I self-host LLM observability for free?

Yes. OpenLIT is free to self-host with Docker Compose or Kubernetes Helm (OpenLIT, ClickHouse, and an OpenTelemetry Collector) with no license key or per-trace fee.

Which open-source LLM observability tool uses OpenTelemetry?

OpenLIT and OpenLLMetry are OpenTelemetry-native. Phoenix uses OpenTelemetry through OpenInference. Langfuse accepts OpenTelemetry traces from v3.

Related guides

Get started

Ready to use the OpenLIT UI in production?

Self-host or connect your stack in minutes. Same harness UI for traces, dashboards, prompts, and evaluations.