Magnitude
Magnitude is an open-source inference engine that automatically tunes its kernels to a user's specific hardware, delivering over 2x faster decode speeds than Ollama on both Apple Silicon and NVIDIA systems. It uses multi-drafter speculative decoding and offers one-click integration with popular AI agent harnesses like Claude Code and Codex, plus an OpenAI-compatible API.
- Artificial Intelligence
- Developer Tools
- Software Only
Funding
Raised to date
$500KRaised to date based on public sources. This may differ from the amount the company actually raised and is based only on what is publicly available on the internet.
Founders
Founder details are not available yet.
Product
Problem
Running large language models locally is often slow and inefficient because generic inference engines apply the same optimization approach to every machine, ignoring the specific capabilities of each GPU or CPU. This poor utilization leads to slow token generation, bottlenecks in prefill and decode phases, and diminished AI agent performance on consumer hardware.
Solution
Magnitude is an open-source inference engine that automatically tunes its kernels on the user's hardware to discover the fastest execution path for each model. It employs advanced techniques like speculative decoding with custom drafters (DFlash, MTP) to accelerate both prefill and decode stages. The engine benchmarks itself against the user's specific configuration and reveals consistent speedups across popular models including Qwen 3.5 4B, Gemma 4 12B, and Nemotron 3.5. With one-click integration for AI agent harnesses and a drop-in OpenAI-compatible API, Magnitude delivers a plug-and-play performance upgrade for local LLM workflows.
Target Audience
Primary users are developers and AI practitioners running local language models for agent-based workflows, application development, or research, who need maximized inference performance on Apple Silicon (M-series) or NVIDIA-based hardware. The product targets both individual power users and teams integrating local AI into development pipelines.
Features
- Hardware-aware kernel tuning technology that optimizes compute kernels for the user's specific GPU/CPU architecture
- Multi-drafter speculative decoding using DFlash and MTP strategies to accelerate token generation
- 2.19x faster decode and 1.32x faster prefill average speedup vs Ollama (MLX) on Mac M4 Pro, and 2.17x decode / 1.34x prefill on NVIDIA DGX Spark
- One-click standalone integration for agent harnesses: Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline
- OpenAI-compatible API for seamless connectivity with any other tool or developer application
- Cross-platform support for macOS, Linux, and Windows
- Open source benchmark harness and evaluation framework for reproducible speed testing