OneTriangle
OneTriangle.ai provides an inference platform that separates prefill from decode by building KV caches with a smaller model and transferring them to a larger model for generation, cutting time-to-first-token by up to 6.3× and prefill costs by 84%. The company offers hosted endpoints for models like Llama 3.3 70B Fast and DeepSeek V4 Flash, with per-token pricing and cached-input discounts.
- Artificial Intelligence
- Software Only
- AI Agents
- Developer Tools
Funding
Raised to date
$500KRaised to date based on public sources. This may differ from the amount the company actually raised and is based only on what is publicly available on the internet.
Founders
Founder details are not available yet.
Product
Problem
Large language models incur high prefill costs because reading prompt context scales with the size of the model performing the read, even before generation begins. This makes serving long-context requests expensive and slow, particularly when a large model must process a long input before producing a single output token. The cost and latency problem is amplified for high-volume production workloads that repeatedly process substantial prompts.
Solution
OneTriangle.ai separates prefill from decode by building KV caches with a smaller, cheaper model and transferring that cache into a larger target model for generation. A learned mapper projects the small model's K/V tensors into the target model's attention space, while a residual correction layer repairs quality gaps. This "prefill small, decode large" approach cuts time-to-first-token by up to 6.3× and reduces prefill cost by 84% for models like Llama 3.1 8B to 70B. Transfer quality is gated on held-out evaluations, and the system falls back to ordinary prefill when a model pair does not pass quality or latency thresholds. The company also offers hosted endpoints and a DFlash parallel drafting mechanism that predicts 16-token blocks in a single forward pass.
Target Audience
Primary customers are AI engineering teams operating production inference workloads at scale, including enterprises using LLM APIs for assistants, retrieval, tool use, and audio applications; the pricing model targets high-volume users needing cost-effective long-context processing.
Features
- KV-cache transfer mechanism that maps a small model's attention tensors into a larger target model's geometry, eliminating redundant prefill computation; the system strips source rotary embeddings, projects head-wise, and re-embeds at target positions
- Learned ridge mapper plus rank-16 residual MLP corrections that preserve task quality, demonstrated with 91.2% quality retention and up to 8.7× faster prefill handoffs
- DFlash parallel drafting that predicts 16-token blocks in a single forward pass, achieving 2.154× geometric-mean decode speedup with exact token-ID agreement against native decoding in all 12 confirmation runs
- Sealed test-set evaluation discipline using SHA-256 pre-registration, frozen checkpoints, and checksum-verified serving bridges to avoid overfitting in reported benchmarks
- Audio-specific KV transfer for voice models like Ultravox, enabling 31.2% accuracy versus 17.2% native 1B at comparable decode speeds of 133-150 tokens per second
- Production endpoints with per-token pricing and cached-input discounts, including DeepSeek V4 Flash, Llama 3.3 70B Fast with 1.61× measured decode improvement, and Vision/action-policy models in limited beta