OpenLIT: the open-source agent harness engineering platform

Trace, evaluate, guard, and improve AI agents and coding agents with OpenTelemetry. Agent observability · Evals · Guardrails · Prompt management · Cost & GPU monitoring · Self-host free (Apache 2.0)

⭐ Star     🚀 Quickstart     📚 Docs

Website · Documentation · Quickstart · Compare · Join Slack

---

What is OpenLIT?

OpenLIT is an open-source agent harness engineering platform. It gives teams OpenTelemetry-native tracing, evaluations, guardrails, prompt and context management, and cost and GPU monitoring for the harness around their AI agents and coding agents, so every agent failure can be traced, scored, and turned into a harness fix. OpenLIT is free to self-host under Apache 2.0 and works with any model, framework, or harness: Claude Code, Codex, Cursor, OpenAI Agents SDK, LangGraph, CrewAI, and 70+ more integrations.

What is agent harness engineering?

An AI agent is a model plus a harness. The harness is everything except the model: tools, context, prompts, memory, hooks, guardrails, and feedback loops. Agent harness engineering is the discipline of designing, measuring, and improving that harness so agents are reliable in production. The core loop is run → observe → evaluate → fix the harness → verify, and OpenLIT gives you each step:

Harness engineering step	OpenLIT feature
Observe every LLM call, tool call, MCP request, retrieval and agent step	Agent observability & OpenTelemetry LLM tracing
Evaluate quality, safety and cost on real traces	Evals (LLM-as-a-judge, programmatic, human feedback) + CI gates
Guard the agent at runtime	Guardrails (prompt injection, sensitive topics, topic restriction)
Fix the harness without redeploying	Prompt Hub, Context, Rule Engine, Vault (secrets)
Compare models and prompts before shipping	OpenGround
Account for cost, latency and hardware	Cost tracking, custom pricing, GPU monitoring (NVIDIA, AMD, Intel)

A production agent can involve:

flowchart TD
    U([User]) --> A[AI Agent]
    A --> L[LLM calls]
    A --> T[Tool calls]
    A --> R[Retrieval]
    A --> M[Memory]
    A --> S[Sub-agents]
    A --> P[Prompts]
    A --> C[Code changes]
    L & T & R & M & S & P & C --> E{{Evaluation}}
    E --> O[["Cost / Quality / Errors"]]

    style U fill:#F97316,stroke:#7C2D12,color:#fff
    style A fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style E fill:#111827,stroke:#F97316,stroke-width:2px,color:#fff
    style O fill:#F97316,stroke:#7C2D12,color:#fff

---

⚡ Get started in 5 minutes

1. Start OpenLIT

git clone https://github.com/openlit/openlit.git
cd openlit

docker compose up -d

Open:

http://127.0.0.1:3000

2. Install the SDK

Python:

pip install openlit

TypeScript:

npm install openlit

3. Instrument your application

Python:

import openlit

openlit.init()

That's it.

OpenLIT automatically instruments supported LLM providers, frameworks, vector databases, and other AI infrastructure and exports OpenTelemetry traces and metrics.