An on-device Android AI assistant powered by VLM, supporting local model inference and screen-level automated interaction.
OpenOmniBot is an on-device Android AI assistant developed by OmniMind. Its core capability leverages Vision-Language Models (VLM) to understand and interact with mobile interfaces, enabling automated App operations, gesture execution, and system settings manipulation.
The project adopts a hybrid architecture combining native Android Kotlin and Flutter, targeting Android 11 through Android 14. The automation engine is built on a state machine pattern managing task lifecycles (Companion, Learning, Scheduled task types), utilizing Android Accessibility Services for screen perception and control. The data layer employs Room database with Flow reactive observation and MMKV lightweight storage.
Inference capabilities are delivered in two editions: a standard edition relying on external model services, and a full edition (omniinfer) integrating both MNN and llama.cpp inference backends for on-device local inference and offline operation. The memory system provides dual-layer short-term and long-term memory with Embedding vectorization support.
For extensibility, the project includes a Skills system (installable via repository links), MCP (Model Context Protocol) server integration, an Alpine Linux environment, an embedded terminal (ReTerminal), and a built-in browser with file workspace. A sub-agent mechanism allows delegating complete tasks to independent sub-agents. System-level integration covers alarms, calendar, audio playback control, and scheduled task execution.
The build system supports Gradle multi-variant configurations (develop/production), allowing developers to choose between the standard edition and the omniinfer full edition. Environment requirements include Flutter SDK 3.9.2+, JDK 11+, and Android SDK (Compile SDK 36, NDK ARMv7/ARM64).
The project uses segmented dual licensing: non-commercial/personal/educational/research use follows GNU AGPL v3, while commercial use requires a commercial license via omnimind.com.cn. The latest version is v0.3.9 with 50 releases and 813+ commits. The OmniMind product line includes OmniInfer, OmniStudio, OmniBot, OmniFlow, OmniRouter, OmniClaw, and OmniCloud.