trytrust.ai is an inference technology company that cuts the cost and latency of large language model serving by replacing expensive prefill compute with a cheaper, faster alternative. Its proprietary KV-cache transfer technique lets a small model read the input and hand off the context to a larger model, while its DFlash parallel drafting accelerates token generation. The company offers these optimizations through a pay-as-you-go API platform hosting models like DeepSeek V4 Flash and Llama 3.3 70B Fast, with measured speedups of up to 2.15× and cost reductions of up to 84% on prefill.
Funding
Funding not disclosed
Founders
Product
Problem
Large language model inference is dominated by the cost of prefill—the phase where a model reads the input. Because prefill costs scale with the size of the model doing the reading, users pay large-model rates before the model produces a single word, making long-context requests disproportionately expensive. Additionally, decode speed is limited by serial next-token prediction, which constrains perceived latency and throughput.
Solution
trytrust.ai lowers inference cost and latency by separating the prefill and decode phases across different models. Its core technology transfers KV caches from a small "reader" model to a larger "decoder" model, allowing the small model to build the context while the large model generates tokens at its native speed. The transfer uses a learned ridge mapper and a rank-16 residual MLP to project the cache into the target model's attention space, gating the handoff on held-out quality metrics to ensure correctness. trytrust.ai also offers DFlash, a parallel drafting technique that predicts 16-token blocks in one forward pass, delivering exact token-matching speedups of over 2× without altering the target model's output. These technologies are served through a unified inference API platform.
Target Audience
Primary customers are AI engineering teams and enterprises seeking low-cost, high-throughput inference for production workloads, including RAG pipelines, real-time voice applications, and safety-classification tasks.
Features
- KV-cache transfer that maps a small model's prefill cache into a larger model's attention space, enabling fast and low-cost long-context handling with gated quality checks
- DFlash parallel drafting that predicts 16-token blocks in a single forward pass, achieving 2.154× geometric-mean speedup with exact token agreement against native decoding
- Handoff overhead of only 68-100 ms per request after warmup, with decode speeds of 133-150 tokens per second for transfer paths
- Model pairing system that falls back to ordinary prefill if a quality or latency gate is not cleared, ensuring reliability
- Audio-specific KV transfer for voice models, improving accuracy from 17.2% to 31.2% on a sealed spoken MMSU test at nearly the same decode speed
- Unified API across text, video, and action-policy inference, including safety classifiers and vision models
- Token-based pricing with no subscriptions, including discounted flex tier at 40% lower cost for delayed delivery