Self-hosted AI coding agent registry with built-in observability telemetry pipeline and LLM-driven evaluation engine, featuring zero-intrusion MCP server call-chain tracing.
Observal is a self-hosted AI coding agent registry positioned as "Docker Hub for AI agents." Its core consists of three capabilities:
Agent Registry: Packages complete agent configurations (MCP servers, skills, hooks, prompts, sandboxes) in a portable YAML format, enabling one-command installation to Claude Code, Kiro CLI, Cursor, Gemini CLI, and other mainstream IDEs for agent discovery, sharing, and reuse.
Telemetry Pipeline: Intercepts communication between IDEs and MCP servers through a transparent shim layer, converting each tool call into a span, which composes traces and sessions. Data flows through the OpenTelemetry Collector into ClickHouse with zero intrusion and zero code modification to observed MCP servers. Supports native OTLP export dual-channel for Claude Code and Kiro.
Evaluation Engine: Automatically scores agent sessions using AWS Bedrock or OpenAI-compatible LLMs across dimensions including goal completion, tool efficiency, factual grounding, thought process, and adversarial robustness, with cross-version comparison support.
IDE Compatibility: Claude Code (Full), Kiro CLI (Full), Gemini CLI, Codex CLI (native OTEL + shim), GitHub Copilot, OpenCode, Cursor (MCP + shim).
Architecture: An 8-service Docker Compose orchestration—Next.js 16 frontend (React 19 + Tailwind CSS 4 + shadcn/ui), FastAPI + Strawberry GraphQL backend (Python 3.11+), PostgreSQL 16 (registry metadata), ClickHouse (telemetry data), Redis + arq (async tasks), OpenTelemetry Collector, Grafana (built-in dashboards), Nginx reverse proxy. Fully self-hosted with no data egress risk. The CLI is distributed as a standalone binary, supporting workflow commands like scan, doctor patch, and pull.
Private Branch Features: The observal-insight-god branch includes Session Reconciliation (parsing Claude Code JSONL files to upload full session data) and Agent Insights V2 (multi-model LLM report generation: Opus narrative, Sonnet synthesis, Haiku facet extraction, with subagent parent-child session attribution tracking), not yet merged into main.
Enterprise Edition: The ee/ directory contains SSO, SCIM, and audit logging capabilities; release status is unconfirmed. Open-sourced under Apache License 2.0, current version v0.3.4.