An open-source Python framework for real-time voice and multimodal conversational AI agents, enabling end-to-end streaming voice interaction via composable Pipeline architecture.
Pipecat is a voice-first multimodal conversational AI agent framework. Its core adopts a composable Pipeline architecture based on Processors, where data (audio frames, text, control messages) flows asynchronously between Processors, supporting end-to-end real-time voice conversation.
The framework covers the full interaction pipeline: integrating 20+ STT, 25+ LLM, and 30+ TTS services, supporting Speech-to-Speech (OpenAI Realtime, Gemini Multimodal Live, etc.), with built-in ONNX-based Smart Turn Detection for local turn analysis. The transport abstraction layer supports WebRTC (Daily, LiveKit), WebSocket, and local audio protocols, with telephony serializers for Twilio, Vonage, etc., and native WhatsApp Transport.
For multimodality, it supports audio, video, and image I/O, connecting to HeyGen, Tavus, and Simli for digital human conversations. Conversation management is handled through Pipecat Flows for structured state machines, and Pipecat Subagents for distributed multi-agent collaboration via shared message buses.
The tooling ecosystem is comprehensive: Pipecat CLI for scaffolding and cloud deployment, Whisker for real-time debugging, Tail for terminal monitoring, and client SDKs for JavaScript, React, Swift, Kotlin, C++, and ESP32. All third-party service dependencies are installed on-demand via extras, keeping the core lightweight.
Requirements: Python ≥ 3.11 (≥ 3.12 recommended), BSD-2-Clause license, latest version v1.1.0.
Quick Start:
uv init my-pipecat-app && cd my-pipecat-app
uv add "pipecat-ai[anthropic,daily,deepgram,openai]"
cp env.example .env
pipecat init quickstart
Unconfirmed: Pipecat Cloud pricing and deployment details are not public; full Speech-to-Speech integration status requires consulting individual service docs; commercial entity information is not explicitly stated in the repo; no quantitative performance benchmarks (e.g., end-to-end latency) available.