screenpipe

Core Positioning#

screenpipe is an open-source, local-first desktop application designed to turn your computer into a personal AI with long-term memory. At its core, it uses an event-driven mechanism to intelligently capture screen screenshots and accessibility trees by listening to OS events (e.g., app switches, clicks, typing pauses, scrolling, clipboard). It automatically falls back to OCR when accessibility APIs are unavailable, keeping storage overhead extremely low (~300 MB/8hr). For audio, it simultaneously captures system audio and microphone input, performing real-time transcription and speaker diarization via a local Whisper model or an optional Deepgram cloud service.

Search & Interaction#

• AI-Powered Search: Natural language search across all OCR text and audio transcripts, with multi-dimensional filtering (app name, window title, browser URL, date range) and embedding-based semantic search, returning associated screenshots and audio clips.
• Timeline View: DVR-like visual retrospection of full-day screen history; click any moment to view the complete screenshot and extracted text.

Extensibility#

• MCP Server: Runs as a standard MCP Server with zero-configuration integration into Claude Desktop, Cursor, VS Code (Cline, Continue), and other mainstream AI clients, providing real-time screen context.
• Pipes Plugin System: Timed AI Agents defined as Markdown files with YAML frontmatter for deterministic data permission control (app filtering, content type, time range, endpoint access), featuring three-layer security enforcement (skill gating, agent interception, server middleware with per-pipe encrypted tokens).
• Apple Intelligence Integration: Leverages Apple Intelligence for local AI processing on supported Mac hardware.

Privacy & Security#

• 100% local storage by default (SQLite), no account registration required
• Fully local AI inference support (Ollama or any OpenAI-compatible model server)
• Optional encryption at rest and end-to-end encrypted device sync
• Sensitive content filtering (by window, app, Chrome extension, passwords, PII)
• Core code MIT-licensed and fully auditable

Architecture#

• Capture Layer: Event-driven — OS events → screenshots + accessibility trees (same timestamp) → OCR fallback when accessibility unavailable
• Audio Processing: Whisper (local) or Deepgram (cloud) → text + speaker labels
• Storage: Local SQLite + FTS5 full-text search; screenshots stored as JPEG
• API Layer: REST API on localhost:3030 covering search, frames, audio, elements, health checks, and Pipe management
• UI Layer: Tauri framework (Rust backend + TypeScript frontend)
• Code Structure: crates/ (Rust core), apps/screenpipe-app-tauri/ (desktop app), packages/ (JS packages), ee/ (enterprise edition)

Use Cases#

• Knowledge Workers & Researchers: Cross-app, cross-time retrieval of previously viewed or heard information
• Developers: Providing real-time and historical screen context to AI coding assistants
• ADHD Users & Remote Workers: Automatic tab/document tracking, automated meeting transcription and notes
• Teams & Enterprises: Cross-organizational deployment of AI Agents with deterministic data permission isolation

Installation#

System Requirements: Min 8 GB RAM, ~5–10 GB disk space/month, 5–10% CPU usage

Desktop App: Download native installers (macOS .dmg / Windows .exe) from the official website

CLI:

npx screenpipe@latest record
claude mcp add screenpipe -- npx -y screenpipe-mcp

SDK Example:

import { pipe } from "@screenpipe/js";
const results = await pipe.queryScreenpipe({
  q: "project deadline",
  contentType: "all",
  limit: 20,
  startTime: new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString(),
});

Platform Support#

• macOS (Apple Silicon & Intel), Windows 10/11 (native installers)
• Linux (source compilation only; no native packages yet)

Unconfirmed Information#

• Enterprise edition (ee/ directory) specific license terms
• Linux native installer release timeline
• SOC 2 / HIPAA compliance certification status
• Version release cadence and update strategy

Competitors#

Rewind.ai (now Limitless), Microsoft Recall, Granola, Otter.ai