Therml builds custom silicon for large‑language‑model inference that embeds the stochastic inference equations directly into the hardware, removing the traditional memory‑bus and its associated energy waste.
Funding
Funding not disclosed
Founders
Product
Problem
Current large language model (LLM) inference relies on conventional GPUs that consume large amounts of energy, primarily due to memory‑bus data movement, making scaling of AI services costly and environmentally unsustainable.
Solution
Therml designs custom silicon architectures that align the stochastic nature of modern AI with the physical behavior of the hardware, eliminating the memory‑bus bottleneck. By embedding the inference equations directly into the circuit, the platform delivers token throughput exceeding 500,000 tokens per second while consuming near‑zero power on the data path. This approach reduces the energy footprint of LLM inference by orders of magnitude compared with standard GPU solutions, enabling high‑performance AI services at lower operational cost and without the massive power overhead of existing infrastructure.
Target Audience
Therml’s primary customers are cloud providers, data center operators, and enterprises that run large language model inference at scale and seek to lower energy consumption and infrastructure costs.
Features
- Specialized compute substrate that matches stochastic AI workloads, removing the need for a traditional memory bus
- Near‑zero power consumption on the data path, achieving sub‑watt operation for large‑scale inference
- Token throughput >500K tokens/second, comparable to high‑end GPUs with a fraction of the energy use
- Architecture treats inference as a natural physical process, integrating equations directly into silicon
- Designed for large language model inference workloads, offering deterministic performance at reduced cost