D-Matrix has developed Corsair, an AI inference platform that achieves 60,000 tokens per second with 1 ms latency for Llama3 8B models, significantly enhancing throughput and energy efficiency in datacenters. This technology addresses the high computational costs and energy consumption associated with large-scale AI inference, enabling organizations to scale their AI capabilities sustainably.
Funding
$161.3M raised to dateRaised to date based on public sources. This may differ from the amount the company actually raised and is based only on what is publicly available on the internet.
+4Founders
Product
Problem
Large language model (LLM) inference in datacenters faces challenges related to computational cost, energy consumption, and latency, hindering the scalable deployment of AI applications. Existing solutions often struggle to deliver the required throughput and real-time performance for interactive AI experiences.
Solution
D-Matrix's Corsair is an AI inference platform designed to address the bottlenecks in large-scale AI deployment. Leveraging a Digital In-Memory Compute (DIMC) architecture, Corsair accelerates AI inference workloads, achieving high token generation rates with low latency. The platform's architecture tightly integrates memory and compute, overcoming memory bandwidth limitations and enabling efficient scaling. Corsair delivers performance, cost savings, and energy efficiency compared to traditional GPU-based solutions.
Target Audience
The primary target audience includes enterprises, OEMs, and system integrators deploying generative AI applications in datacenters, particularly those seeking to improve performance, reduce costs, and enhance energy efficiency.
Features
- Digital In-Memory Compute (DIMC) architecture for tightly integrated memory and compute
- DMX Link for high-speed, energy-efficient die-to-die connectivity across chiplets
- DMX Bridge for connecting packages across two cards
- Native support for block floating point numerical formats (Micro-scaling MX)
- Aviator software stack for a familiar user experience and tooling
- Industry-standard PCIe Gen5 full height, full length card form factor
- Up to 2400 TFLOPs of 8-bit peak compute per Corsair card
- Up to 256 GB of off-chip Capacity Memory
- Memory bandwidth of 150 TB/s