RunAnywhere provides MetalRT, an on‑device AI inference engine that runs large language, speech, and vision models directly on Apple Silicon and Android hardware. By using hand‑crafted GPU kernels, operator fusion, and unified memory optimization, it delivers low‑latency, high‑throughput inference without cloud dependence, while a cross‑platform SDK and control‑plane simplify model deployment, updates, and telemetry for mobile and edge developers.
Funding
Funding not disclosed
Founders
Product
Problem
Cloud-based AI inference incurs high latency and recurring compute costs, making real-time, privacy-sensitive applications impractical at scale, especially for billions of end‑users and regulated industries.
Solution
RunAnywhere delivers a native on‑device AI inference engine called MetalRT that runs large language models, speech‑to‑text, text‑to‑speech, and vision‑language models directly on Apple Silicon and Android hardware. By hand‑crafting GPU kernels, fusing operators, and optimizing unified memory, MetalRT achieves record‑low latency (e.g., 6.6 ms time‑to‑first‑token, 101 ms transcription of 70 s audio) and high throughput (up to 658 tokens/s decode) without any cloud dependency. The company provides a cross‑platform SDK (Swift, Kotlin, React Native, Flutter) and a control‑plane service that manages model versioning, OTA updates, and fleet‑wide observability, allowing developers to integrate on‑device AI with the same simplicity as a cloud API while preserving speed, privacy, and reliability.
Target Audience
Primary customers are mobile app developers, enterprise product teams, and regulated‑industry vendors (e.g., healthcare, finance, telecom) that need real‑time, privacy‑preserving AI capabilities on iOS, Android, or edge devices.
Features
- Hand‑written Metal Shading Language kernels and operator fusion for maximum performance on Apple Silicon GPUs
- Unified C++ runtime (MetalRT) that supports LLM, speech‑to‑text, text‑to‑speech, and vision‑language models on a single device
- Cross‑platform SDK with bindings for iOS (Swift), Android (Kotlin), React Native, and Flutter
- Fleet‑scale control plane for model distribution, over‑the‑air updates, and performance telemetry
- Quantization‑aware execution (4‑bit models) to fit large models in limited device memory
- Benchmark‑proven speedups: 1.5‑2.4× faster than Apple MLX, llama.cpp, and other open‑source engines across all modalities
- Offline‑first operation guaranteeing zero data leaving the device, meeting privacy and regulatory requirements