# What is llm observability?

> LLM observability is the practice of tracing, measuring, and evaluating large language model applications in production: prompts and completions, token usage, cost, latency, errors, and output quality.

- HTML: https://openlit.io/glossary/llm-observability
- Markdown: https://openlit.io/glossary/llm-observability.md

Traditional APM tells you a request was slow or failed. LLM observability adds model-specific signals: which prompt and model produced a response, how many tokens it used, what it cost, and whether the output was accurate, relevant, and safe.

Core metrics are latency (including time to first token), token usage, cost per request, error and rate-limit rates, and quality scores from evaluations such as LLM-as-a-judge. For RAG apps, retrieval quality and vector database latency matter too. For agents, observability extends to tool calls and full trajectories (see agent observability).

OpenTelemetry GenAI semantic conventions standardize how LLM spans and metrics are recorded. OpenLIT is an open-source LLM observability platform that emits those spans natively, stores them in ClickHouse, and can export the same data to Grafana, Datadog, or any OTLP backend.

## FAQ

### What is an LLM observability platform?

An LLM observability platform collects traces and metrics from LLM applications (prompts, responses, tokens, cost, latency, and errors) and adds evaluation, so teams can debug and improve quality in production. OpenLIT is an open-source, OpenTelemetry-native LLM observability platform.

### What metrics matter for LLM observability?

Latency and time to first token, token usage, cost per request and per user, error and rate-limit rates, and quality scores such as relevance, hallucination, and toxicity.

### What are the top LLM observability platforms?

OpenLIT, Langfuse, Arize Phoenix, LangSmith, Comet Opik, Braintrust, Helicone, Datadog LLM Observability, MLflow, and OpenLLMetry. See the comparison at https://openlit.io/guides/best-llm-observability-tools.

## Related

- [Agent observability](https://openlit.io/glossary/agent-observability.md)
- [Agent evals](https://openlit.io/glossary/agent-evals.md)
- [Guardrails](https://openlit.io/glossary/guardrails.md)
- Pillar: https://openlit.io/agent-harness-engineering.md
