# Open source Harness Engineering Platform > Trace LLMs and AI agents with OpenTelemetry, run LLM evaluations, manage prompts, compare models, store API keys in Vault, and self-host free under Apache 2.0. - Site: https://openlit.io - Docs: https://docs.openlit.io/latest/openlit/quickstart-ai-observability - GitHub: https://github.com/openlit/openlit - License: Apache 2.0 - Markdown: https://openlit.io/index.md ## What OpenLIT is OpenLIT is an open source Harness Engineering platform and an open source AI engineering platform. Teams use it to instrument AI apps, manage prompts, run LLM evaluations, monitor agents, store API keys in Vault, and improve quality with production data. It is built on OpenTelemetry and free to self-host under Apache 2.0. ## Instrumentation paths 1. **Native SDKs** (Python, JavaScript, Go): `pip install openlit` / `npm install openlit` / `go get github.com/openlit/openlit/sdk/go`, then `openlit.init()`. 2. **eBPF controller** (enterprise): zero-code instrumentation for Kubernetes, Docker, and Linux. 3. **GPU collector**: OpenTelemetry GPU metrics for NVIDIA, AMD, and Intel. 4. **Any OTel source**: point OTel SDKs, OBI, OpenLLMetry, or other OTLP exporters at OpenLIT. OpenLIT has 56+ integrations across LLMs, agent frameworks, vector databases, and GPUs. ## FAQ ### What is OpenLIT? OpenLIT is an open source Harness Engineering platform. Teams use it to instrument AI apps, manage prompts, run LLM evaluations, monitor agents, store API keys in Vault, and improve quality with production data. It is built on OpenTelemetry and free to self-host under Apache 2.0. ### What is harness engineering? Harness engineering is the loop of instrument, develop, manage, observe, and improve AI systems. OpenLIT is the open source Harness Engineering platform for that loop, and also an open source AI engineering platform for tracing, prompts, evaluations, and secrets. ### Does OpenLIT support LLM tracing and OpenTelemetry? Yes. OpenLIT provides OpenTelemetry-native LLM tracing and metrics. Use OpenLIT SDKs for Python, JavaScript, and Go, or send OTLP from OTel SDKs, OBI, OpenLLMetry, and other OpenTelemetry sources. ### Can I run LLM evaluation and LLM-as-a-judge in OpenLIT? Yes. OpenLIT supports LLM evaluation with LLM-as-a-judge, heuristics, and human review. You can score production traffic or experiments, including relevance, accuracy, safety, and RAG quality signals. ### Does OpenLIT include prompt management? Yes. Prompt Hub lets you version prompts, deploy updates, and roll back without shipping app code every time. That keeps prompt management and prompt engineering in one place for the team. ### Can I compare LLM models side by side? Yes. OpenGround is a model comparison playground. Test prompts on real inputs and compare models side by side before you ship. ### Does OpenLIT support AI agent monitoring? Yes. OpenLIT supports AI agent monitoring and coding agent workflows. Trace agent steps, tool calls, cost, and outcomes, including Claude Code style coding agents. ### How does Vault help with API key management? Vault stores and rotates LLM API keys and secrets so they stay out of application code. Use it for OpenAI, Anthropic, and other provider keys with central API key management. ### Does OpenLIT monitor GPU utilization for LLM inference? Yes. The OpenLIT GPU collector exports NVIDIA, AMD, and Intel metrics such as GPU utilization, memory, temperature, and power as OpenTelemetry signals you can correlate with inference traces. ### Is OpenLIT a good open source AI engineering platform? Yes. OpenLIT is an open source Harness Engineering platform and an open source AI engineering platform under Apache 2.0, with native SDKs for Python, JavaScript, and Go, plus 56+ integrations. You can self-host with Docker or Kubernetes, keep data in your own ClickHouse, and export OTLP to tools like Grafana or Datadog. ### How do I deploy OpenLIT? Self-host with Docker Compose or Kubernetes Helm. The stack is OpenLIT, ClickHouse, and an OpenTelemetry Collector. You can also reuse your own ClickHouse and collector. ### Is self-hosting OpenLIT free? Yes. Self-hosted OpenLIT is free under Apache 2.0. There is no license key and no per-trace fee. You only pay for the servers you run. ### Does OpenLIT add latency to LLM calls? OpenLIT instruments in process and exports data in the background over OpenTelemetry. There is no required proxy in front of every LLM call. ### Why does this website look like this? Because our product looks like this. We liked the UI so much we let it escape the dashboard and take over the marketing site. Same sidebar energy, same Otter vibes, fewer traces (for now). ### How do I get started with OpenLIT? Run docker compose up -d to start the platform. Then install the SDK with pip install openlit and call openlit.init(), or point any OTLP exporter at your OpenLIT endpoint. ## Related - Pricing: https://openlit.io/pricing.md - About: https://openlit.io/about-us.md - Compare: https://openlit.io/compare.md - Product docs (llms.txt): https://docs.openlit.io/llms.txt --- # OpenLIT Pricing: Free Open Source Self-Host and Cloud > OpenLIT pricing is simple. Self-host the open source Harness Engineering platform free under Apache 2.0 with unlimited usage. OpenLIT Cloud is coming soon for fully hosted teams. - HTML: https://openlit.io/pricing - Markdown: https://openlit.io/pricing.md ## Plans ### OSS (Available now) Self-host the open source Harness Engineering platform on your own infra with Helm or Docker. - Price: Free. Apache 2.0. Unlimited self-hosted usage. - CTA: [Deploy from GitHub](https://github.com/openlit/openlit) - Docs: [Documentation](https://docs.openlit.io/latest/openlit/quickstart-ai-observability) Highlights: - Apache 2.0 licensed OpenLIT platform - Full stack AI infra monitoring for GPUs, LLMs, MCP, vector DBs, and coding agents - Core platform features and APIs for tracing, evaluation, prompts, and more - Deployment docs and Helm chart - OAuth sign-in - Community support on GitHub ### Cloud (Coming soon) Fully hosted OpenLIT for teams that prefer managed operations. - Price: Coming soon. Join the waitlist for launch updates and pricing. - CTA: [Join waitlist](mailto:waitlist@openlit.io?subject=OpenLIT%20Cloud%20Waitlist) Highlights: - Fully hosted OpenLIT platform - Managed infrastructure and upgrades - Built for teams that prefer zero ops - Feature set and pricing shared at launch ## OSS features included ### Zero-code instrumentation eBPF and SDK auto-instrumentation across Kubernetes, Docker, and Linux. - Controller binary and deployment options: Yes ### Coding agent observability Session, cost, and code-impact tracking for coding agents. - Session, cost, token, and repo activity tracking: Yes ### Fleet Hub OpenTelemetry collector monitoring and config via OpAMP. - Single-cluster collector monitoring: Yes ### Prompt Hub Prompt versioning, retrieval, and AI-assisted improvement. - Prompt versioning: Yes - Dynamic variables and API retrieval: Yes - Prompt fetching: Unlimited - AI-assisted prompt improvement: Yes ### Vault Encrypted secret storage for LLM API keys. - Key and tag based retrieval: Yes - API authentication: Yes - Encrypted storage (AES-256-GCM): Yes ### OpenGround Side-by-side LLM comparison for latency, tokens, and cost. - BYOK model comparison: Yes ### Custom dashboards ClickHouse-backed dashboards with charts, tables, and filters. - Dashboard creation, widgets, and folders: Yes - Import and export: Yes ### Evaluations Built-in evaluation types with rule-engine automation. - Built-in evaluation types: Yes - Programmatic and local or CI execution: Yes - Custom eval prompts and rule-based selection: Yes ### Guardrails Prompt-injection, jailbreak, and sensitive-topic detection. - Built-in guards in SDK (LLM and regex): Yes ### GPU monitoring GPU utilization, memory, temperature, and power visibility. - Collector and dashboards: Yes ### Usage and limits - Included usage: Unlimited (self-hosted) - Access to historical data: Unlimited (self-managed) - Users: Unlimited - Projects and environments: Unlimited ### Support - Community support (GitHub): Yes ### Security and compliance - Sign in with Google, Azure AD, GitHub: Yes ## FAQ ### What is OpenLIT pricing? OpenLIT OSS is free to self-host under Apache 2.0 with no license key and no per-trace fee. OpenLIT Cloud is coming soon as a fully hosted option. Cloud pricing will be shared at launch. ### Is OpenLIT free and open source? Yes. OpenLIT is an open source Harness Engineering platform. You can self-host it free forever under Apache 2.0. Deploy with Docker Compose or Kubernetes Helm from GitHub. ### Can I self-host OpenLIT for LLM observability? Yes. Self-host OpenLIT on your own infrastructure. The stack includes OpenLIT, ClickHouse, and an OpenTelemetry Collector. Usage on OSS is unlimited because you run it yourself. ### Is OpenLIT Apache 2.0 licensed? Yes. OpenLIT is Apache 2.0 licensed. You can inspect the code, self-host, modify, and contribute without a paid license for OSS. ### What is OpenLIT Cloud? OpenLIT Cloud is the fully hosted version of OpenLIT for teams that want managed operations instead of running their own stack. Feature and pricing details will be shared at launch. Join the waitlist for updates. ### When will OpenLIT Cloud pricing be available? Cloud is in development. Pricing details will be published at launch. Email waitlist@openlit.io to join the waitlist. ### Does self-hosted OpenLIT support OpenTelemetry? Yes. OpenLIT is OpenTelemetry-native. Use OpenLIT SDKs or send OTLP to OpenLIT, and export traces to Grafana, Datadog, or any OTLP backend. ### What is included in OpenLIT OSS? OSS includes LLM tracing, coding agent observability, Prompt Hub, Vault, OpenGround, evaluations, guardrails, GPU monitoring, Fleet Hub, custom dashboards, OAuth, and community support on GitHub. --- # About OpenLIT > OpenLIT is building the open source Harness Engineering platform to help teams instrument, develop, manage, observe, and improve production AI systems faster. - HTML: https://openlit.io/about-us - Markdown: https://openlit.io/about-us.md - GitHub: https://github.com/openlit/openlit - Contact: contact@openlit.io ## Story We are excited about the potential of LLMs and generative AI, and the impact they will have on how software gets built. Doing our part to accelerate that shift is our mission. OpenLIT started from the pain of shipping LLM apps without clear traces, cost, quality, or agent visibility. The answer was an OpenTelemetry-native AI engineering platform you can self-host free under Apache 2.0: LLM tracing, evaluations, Prompt Hub, Vault, OpenGround, coding agent monitoring, and GPU metrics in one place. Today OpenLIT is used by developers worldwide. The project is open, community-driven, and built in public on GitHub. ## Join us - Contribute: https://github.com/openlit/openlit/blob/main/CONTRIBUTING.md - Docs: https://docs.openlit.io/latest/overview - GitHub: https://github.com/openlit/openlit - Email: contact@openlit.io Live contributor avatars and public GitHub metrics are shown on the HTML About page. --- # OpenLIT vs Alternatives > Compare OpenLIT against Langfuse, Helicone, LangSmith, and Datadog. Honest, feature-by-feature comparison of LLM observability and monitoring tools. - HTML: https://openlit.io/compare - Markdown: https://openlit.io/compare.md ## Comparisons - [OpenLIT vs Langfuse](https://openlit.io/compare/openlit-vs-langfuse.md): Compare OpenLIT and Langfuse for LLM observability. See how OpenTelemetry-native monitoring, GPU tracking, and self-hosted deployment stack up. - [OpenLIT vs Helicone](https://openlit.io/compare/openlit-vs-helicone.md): Compare OpenLIT and Helicone for LLM observability. OpenLIT is SDK-based with no proxy latency, OpenTelemetry-native, and fully self-hostable. - [OpenLIT vs LangSmith](https://openlit.io/compare/openlit-vs-langsmith.md): Compare OpenLIT and LangSmith for LLM monitoring. OpenLIT is open-source, self-hostable, and framework-agnostic. LangSmith is a proprietary cloud platform optimised for LangChain. - [OpenLIT vs Datadog LLM Observability](https://openlit.io/compare/openlit-vs-datadog.md): Compare OpenLIT and Datadog for LLM observability. OpenLIT is purpose-built, open-source, and free to self-host. Datadog is a general-purpose platform with LLM monitoring as a paid add-on. --- # OpenLIT vs Langfuse > Both are open-source LLM observability tools. OpenLIT is built natively on OpenTelemetry standards with GPU monitoring and broader infrastructure coverage. Here is a detailed comparison. - HTML: https://openlit.io/compare/openlit-vs-langfuse - Markdown: https://openlit.io/compare/openlit-vs-langfuse.md Compare OpenLIT and Langfuse for LLM observability. See how OpenTelemetry-native monitoring, GPU tracking, and self-hosted deployment stack up. ## Feature comparison ### Core Architecture | Feature | OpenLIT | Langfuse | | --- | --- | --- | | OpenTelemetry-native | Yes | OTel ingestion added in v3; primary SDK is proprietary | | Open Source | Apache 2.0 | MIT (core) + EE (enterprise) | | Self-hostable | Yes | Yes | | Cloud-managed option | Coming soon | Yes | | Vendor-neutral data export | Yes | Via OTel export (v3+) | ### LLM Monitoring | Feature | OpenLIT | Langfuse | | --- | --- | --- | | Token usage tracking | Yes | Yes | | Cost per request | Yes | Yes | | Latency / p95 metrics | Yes | Yes | | Prompt & response logging | Yes | Yes | | Streaming support | Yes | Yes | | 60+ integrations (LLMs, frameworks, VectorDBs, GPUs) | Yes | Relies on OTel SDKs or manual instrumentation | ### Infrastructure Monitoring | Feature | OpenLIT | Langfuse | | --- | --- | --- | | GPU monitoring (NVIDIA + AMD) | Yes | No | | Vector DB tracing | Yes | No | | Multi-environment tagging | Yes | Yes | | Organisation management | Yes | Yes | ### Developer Tools | Feature | OpenLIT | Langfuse | | --- | --- | --- | | Prompt Hub (versioning) | Yes | Yes | | Evaluations | Yes | Yes | | Secrets Vault | Yes | No | | Fleet Hub (multi-deployment) | Yes | No | | Custom model pricing | Yes | No | ### Observability Backends | Feature | OpenLIT | Langfuse | | --- | --- | --- | | Grafana / Prometheus export | Yes | Via OTel (v3+) | | Datadog export | Yes | No | | Any OTLP-compatible backend | Yes | Via OTel (v3+) | ## When to choose OpenLIT - You need GPU monitoring for self-hosted LLMs (Ollama, vLLM) on NVIDIA or AMD hardware - You want a fully OTel-native stack where every span is a standard OpenTelemetry span - You need Vector DB observability alongside LLM monitoring - You want Vault, Fleet Hub, and 60+ auto-instrumented integrations in one platform - You are already using OpenTelemetry and want data to flow to any OTLP backend ## When to choose Langfuse - You want a mature cloud SaaS with a larger community and dedicated support tiers - Your primary workflow centres on human annotation, labelling, and feedback loops - You need a polished evaluation and dataset management UI with fine-grained scoring ## Related - All comparisons: https://openlit.io/compare.md - Pricing: https://openlit.io/pricing.md - Docs: https://docs.openlit.io/latest/openlit/quickstart-ai-observability --- # OpenLIT vs Helicone > Helicone routes traffic through a proxy to capture telemetry. OpenLIT instruments your existing SDK calls directly — no proxy, no added latency, fully OpenTelemetry-native. - HTML: https://openlit.io/compare/openlit-vs-helicone - Markdown: https://openlit.io/compare/openlit-vs-helicone.md Compare OpenLIT and Helicone for LLM observability. OpenLIT is SDK-based with no proxy latency, OpenTelemetry-native, and fully self-hostable. ## Feature comparison ### Core Architecture | Feature | OpenLIT | Helicone | | --- | --- | --- | | OpenTelemetry-native | Yes | No | | Open Source | Apache 2.0 | Apache 2.0 | | Self-hostable | Yes | Yes | | Proxy-free (no added latency) | Yes | No | | SDK-based instrumentation | Yes | No | | Vendor-neutral data format | Yes | No | ### LLM Monitoring | Feature | OpenLIT | Helicone | | --- | --- | --- | | Token usage tracking | Yes | Yes | | Cost per request | Yes | Yes | | Latency / p95 metrics | Yes | Yes | | Prompt & response logging | Yes | Yes | | 60+ integrations (LLMs, frameworks, VectorDBs, GPUs) | Yes | OpenAI-compatible endpoints only | ### Infrastructure Monitoring | Feature | OpenLIT | Helicone | | --- | --- | --- | | GPU monitoring (NVIDIA + AMD) | Yes | No | | Vector DB tracing | Yes | No | | Multi-environment tagging | Yes | Yes | | Organisation management | Yes | Yes | ### Developer Tools | Feature | OpenLIT | Helicone | | --- | --- | --- | | Prompt Hub (versioning) | Yes | Yes | | Evaluations | Yes | Basic | | Secrets Vault | Yes | No | | Fleet Hub (multi-deployment) | Yes | No | | Custom model pricing | Yes | No | | Request caching | No | Yes | | Rate limiting & key management | No | Yes | ### Observability Backends | Feature | OpenLIT | Helicone | | --- | --- | --- | | Grafana / Prometheus export | Yes | No | | Datadog export | Yes | No | | Any OTLP-compatible backend | Yes | No | ## When to choose OpenLIT - You cannot tolerate proxy-introduced latency on every LLM call - You need GPU monitoring for self-hosted models (NVIDIA or AMD) - You use non-OpenAI-compatible APIs (Anthropic native SDK, Bedrock, Vertex AI, etc.) - You want to forward telemetry to Grafana, Datadog, or any OTLP backend - You need Vector DB observability alongside LLM request tracing ## When to choose Helicone - You want built-in request caching to reduce API costs - You need proxy-layer rate limiting and API key management - You prefer a cloud-first setup with minimal local infrastructure ## Related - All comparisons: https://openlit.io/compare.md - Pricing: https://openlit.io/pricing.md - Docs: https://docs.openlit.io/latest/openlit/quickstart-ai-observability --- # OpenLIT vs LangSmith > LangSmith is LangChain's proprietary observability platform. OpenLIT is framework-agnostic, fully open-source (Apache 2.0), and works with any LLM provider or orchestration framework. - HTML: https://openlit.io/compare/openlit-vs-langsmith - Markdown: https://openlit.io/compare/openlit-vs-langsmith.md Compare OpenLIT and LangSmith for LLM monitoring. OpenLIT is open-source, self-hostable, and framework-agnostic. LangSmith is a proprietary cloud platform optimised for LangChain. ## Feature comparison ### Core Architecture | Feature | OpenLIT | LangSmith | | --- | --- | --- | | OpenTelemetry-native | Yes | No | | Open Source | Apache 2.0 | No | | Self-hostable | Yes | Enterprise plan only | | Framework-agnostic | Yes | Best with LangChain; supports others via SDK | | Free self-hosted tier | Yes | No | ### LLM Monitoring | Feature | OpenLIT | LangSmith | | --- | --- | --- | | Token usage tracking | Yes | Yes | | Cost per request | Yes | Yes | | Latency / p95 metrics | Yes | Yes | | Prompt & response logging | Yes | Yes | | 60+ integrations (LLMs, frameworks, VectorDBs, GPUs) | Yes | Primarily via LangChain wrappers | ### Infrastructure Monitoring | Feature | OpenLIT | LangSmith | | --- | --- | --- | | GPU monitoring (NVIDIA + AMD) | Yes | No | | Vector DB tracing | Yes | No | | Multi-environment tagging | Yes | Yes | | Organisation management | Yes | Yes | ### Developer Tools | Feature | OpenLIT | LangSmith | | --- | --- | --- | | Prompt Hub (versioning) | Yes | Yes | | Evaluations | Yes | Yes | | Secrets Vault | Yes | No | | Fleet Hub (multi-deployment) | Yes | No | | Custom model pricing | Yes | No | | Prompt playground | No | Yes | | Dataset management for evals | No | Yes | ### Pricing | Feature | OpenLIT | LangSmith | | --- | --- | --- | | Free tier | Unlimited traces (self-hosted) | 5,000 traces/month | | No credit card to start | Yes | No | | Self-hosted free tier | Yes | No | | Data ownership | Full (self-hosted) | LangChain cloud | ## When to choose OpenLIT - You are not using LangChain and want framework-agnostic, auto-instrumented observability - You need to self-host for data privacy, compliance, or cost control without an Enterprise plan - You need GPU monitoring for locally-hosted models on NVIDIA or AMD hardware - You want unlimited traces on self-hosted infrastructure at zero licensing cost - You require OpenTelemetry compatibility to route data to Grafana, Datadog, or other backends ## When to choose LangSmith - Your stack is deeply integrated with LangChain and you want first-class toolchain support - You need LangSmith's prompt playground and dataset management for systematic eval workflows - You are comfortable with cloud-only and want a managed service without any ops overhead ## Related - All comparisons: https://openlit.io/compare.md - Pricing: https://openlit.io/pricing.md - Docs: https://docs.openlit.io/latest/openlit/quickstart-ai-observability --- # OpenLIT vs Datadog LLM Observability > Datadog is a powerful general-purpose observability platform, but LLM monitoring is a paid add-on requiring existing Datadog infrastructure. OpenLIT is purpose-built for AI engineering, open-source, and free to self-host. - HTML: https://openlit.io/compare/openlit-vs-datadog - Markdown: https://openlit.io/compare/openlit-vs-datadog.md Compare OpenLIT and Datadog for LLM observability. OpenLIT is purpose-built, open-source, and free to self-host. Datadog is a general-purpose platform with LLM monitoring as a paid add-on. ## Feature comparison ### Core Architecture | Feature | OpenLIT | Datadog LLM Observability | | --- | --- | --- | | OpenTelemetry-native | Yes | OTel ingestion supported; primary agent is proprietary | | Open Source | Apache 2.0 | No | | Self-hostable | Yes | No | | Purpose-built for LLM / AI engineering | Yes | No | | Free self-hosted tier | Yes | No | ### LLM Monitoring | Feature | OpenLIT | Datadog LLM Observability | | --- | --- | --- | | Token usage tracking | Yes | Yes | | Cost per request | Yes | Yes | | Latency / p95 metrics | Yes | Yes | | Prompt & response logging | Yes | Yes | | 60+ integrations (LLMs, frameworks, VectorDBs, GPUs) | Yes | Limited LLM providers supported natively | | Custom model pricing | Yes | No | ### Infrastructure Monitoring | Feature | OpenLIT | Datadog LLM Observability | | --- | --- | --- | | GPU monitoring (NVIDIA + AMD) | Yes | General GPU metrics only, not LLM-context-aware | | Vector DB tracing | Yes | No | | Multi-environment tagging | Yes | Yes | | Organisation management | Yes | Yes | ### Developer Tools | Feature | OpenLIT | Datadog LLM Observability | | --- | --- | --- | | Prompt Hub (versioning) | Yes | No | | Evaluations | Yes | Basic (LLM-as-judge) | | Secrets Vault | Yes | No | | Fleet Hub (multi-deployment) | Yes | No | | Out-of-box LLM dashboards | Yes | Requires setup / add-on | | Unified infra + APM + LLM observability | No | Yes | ### Pricing | Feature | OpenLIT | Datadog LLM Observability | | --- | --- | --- | | Free self-hosted | Yes | No | | No per-host agent fees | Yes | No | | No usage-based billing (self-hosted) | Yes | No | | LLM Observability cost | $0 (self-hosted) | Usage-based paid add-on on top of Datadog subscription | ## When to choose OpenLIT - You want purpose-built LLM observability without the overhead of a general APM platform - You need to self-host for data privacy, compliance, or to avoid cloud vendor costs - You want Prompt Hub, Vault, and Fleet Hub included as part of your AI engineering platform - You prefer open standards (Apache 2.0, OpenTelemetry) over proprietary agents and lock-in - You want GPU monitoring that is context-aware of LLM inference, not just system-level GPU metrics ## When to choose Datadog LLM Observability - You are already invested in Datadog and want a single pane of glass across all infrastructure - You need enterprise-grade SLAs, SOC 2, HIPAA compliance, and dedicated support contracts - You want unified alerting across LLM, APM, infrastructure, and logs in one existing platform ## Related - All comparisons: https://openlit.io/compare.md - Pricing: https://openlit.io/pricing.md - Docs: https://docs.openlit.io/latest/openlit/quickstart-ai-observability