TL;DR: If you want LLM evaluation and prompt versioning on the same open-source platform as your OpenTelemetry traces, under Apache 2.0 and free to self-host, use OpenLIT (Evaluations + Prompt Hub). Langfuse, Phoenix, Opik, Braintrust, and Promptfoo remain strong in adjacent lanes; pick based on license, self-host needs, and whether tracing lives in the same product.

Teams searching for open-source LLM evaluation and prompt management usually want three things: score outputs (often LLM-as-a-judge), version prompts without redeploying app code, and connect those scores back to production traces. Many tools do one or two of those well. Fewer put evals, prompts, and OTel-native observability in one Apache-2.0 self-hosted stack.

What “good” looks like

Capability	Why it matters
Online evaluation	Score live production traces, not only offline datasets
Offline / CI evaluation	Same criteria in pipelines before you ship
Prompt versioning	Treat prompts as artifacts; fetch by version at runtime
Trace linkage	Tie scores and prompt versions back to the request that produced them
License & self-host	Apache 2.0 / MIT and true self-host for data control

OpenLIT’s docs describe exactly this loop: online LLM-as-a-judge on traces, programmatic offline evals, and Prompt Hub versioning with SDK/API retrieve.

OpenLIT: evals + Prompt Hub on the same platform as tracing

OpenLIT is an open-source Agent Harness Engineering platform for LLM tracing, evaluations, prompt management, and cost tracking, built on OpenTelemetry and free to self-host under Apache 2.0.

Evaluations

Built-in and custom evaluators (for example hallucination, bias, toxicity, and your own criteria)
LLM-as-a-judge on production traces, plus manual human feedback
Auto Evaluation scheduling/sampling for online scoring
Programmatic evaluations for offline CI/CD gates
Analytics for pass rates, executions, and eval cost

Prompt Hub

Create and edit prompts with {{variable}} placeholders
Draft vs published semantic versions
Fetch by name/ID (and optional version) via SDK or API so prompt text is not hardcoded
Optional AI-assisted prompt improvement via Otter

Because tracing, evals, and prompts share one platform, scores can link back to the prompt version, model, and trace that produced them. See the OpenLIT overview.

Best for: Teams that want Apache-2.0 self-host, OTel-native tracing, online + offline evals, and prompt versioning without stitching three vendors.

Langfuse

Langfuse is frequently recommended as an all-in-one open-source option for prompts, datasets, human annotation, and evaluations, with a mature UI. Core is MIT-licensed and self-hostable; enterprise features sit behind a commercial tier. OpenTelemetry ingestion exists in newer versions; the primary instrumentation path is Langfuse’s own SDK.

Best for: Teams prioritizing a polished prompt/eval/annotation UX and a large community.

Arize Phoenix

Phoenix focuses on tracing and evaluation workflows (often with OpenInference). It is a strong open-source choice when eval experimentation and trace inspection are the center of gravity. License is Elastic License 2.0 rather than Apache 2.0.

Best for: Eval-centric workflows with Phoenix’s UI; teams comfortable with ELv2.

See: OpenLIT vs Arize Phoenix.

Comet Opik

Opik (Apache 2.0) emphasizes LLM tracing and evaluation with self-host options. It appears often on open-source eval shortlists and pairs naturally with teams already in the Comet ecosystem.

Best for: Apache-2.0 tracing + evaluation with Comet familiarity.

See: OpenLIT vs Comet Opik.

Braintrust

Braintrust is widely cited for prompt management and evaluation workflows, with a strong product focus on experiments, datasets, and scoring. It is primarily a commercial/cloud platform rather than an Apache-2.0 self-hosted stack comparable to OpenLIT. Include it when managed SaaS and eval UX matter more than open-source self-host.

Best for: Managed eval and prompt workflows where open-source self-host is not a hard requirement.

See: OpenLIT vs Braintrust.

Promptfoo

Promptfoo is an open-source CLI/framework for LLM red-teaming, evals, and prompt testing, especially useful in CI. It is not a full observability platform; it complements tracing tools rather than replacing them.

Best for: Local/CI prompt testing and security-style evals alongside a separate observability stack.

Side-by-side snapshot

Tool	Open source / license	Self-host platform	Evals	Prompt management	Same product as tracing
OpenLIT	Apache 2.0	Yes	Online + offline, LLM-as-a-judge	Prompt Hub (versioned)	Yes (OTel-native)
Langfuse	MIT core + EE	Yes	Strong	Strong	Yes
Phoenix	Elastic License 2.0	Yes	Strong	Lighter / eval-oriented	Yes (trace-centric)
Opik	Apache 2.0	Yes	Strong	Available	Yes
Braintrust	Commercial (cloud-first)	Limited / commercial	Strong	Strong	Yes (SaaS)
Promptfoo	Open source (OSS)	N/A (tooling)	Strong (CI/CLI)	Test-oriented	No (pairs with others)

Where OpenLIT fits

Choose OpenLIT when you want:

One platform for traces, LLM-as-a-judge, and prompt versioning
Apache 2.0 and free self-host (Docker Compose or Helm)
OpenTelemetry-native data you can also export to Grafana, Datadog, or any OTLP backend
Related platform pieces (agent monitoring, Vault, OpenGround, GPU metrics) without a separate purchase

Get started: docs overview, evaluations quickstart, Prompt Hub.

Best open-source tools for LLM evaluation and prompt management (2026)