ZML offers a production inference stack that compiles any AI model directly to NVIDIA, AMD, TPU, and Trainium accelerators from a single codebase, eliminating the need for Python runtimes and hardware‑specific rewrites. Its explicit platform API lets engineers control placement, memory, and I/O for deterministic, peak‑performance inference across heterogeneous hardware.
Funding
Funding not disclosed
Founders
Product
Problem
Deploying AI inference workloads across diverse accelerators often requires rewriting code for each hardware target, managing heavyweight Python runtimes, and coping with hidden state and abstraction overhead that obscure performance and predictability.
Solution
ZML provides a production inference stack that compiles models directly to NVIDIA, AMD, TPU, and Trainium accelerators from a single codebase, eliminating the need for Python runtimes and extensive rewrites. The platform exposes explicit abstractions—such as platform objects, pinned memory allocators, zero‑copy I/O, and a user‑land virtual file system—so developers can control compilation, placement, and data movement with predictable performance. By targeting the metal, ZML achieves peak hardware utilization while maintaining composability, allowing complex pipelines (e.g., sharding, quantization) to be expressed without implicit global state. The stack also bundles minimal runtime sandboxes, enabling seamless deployment across heterogeneous environments.
Target Audience
Primary customers are AI engineers and DevOps teams that need high‑performance, hardware‑agnostic inference pipelines for large language models, vision models, or other compute‑intensive workloads in production environments.
Features
- Direct compilation of any model to NVIDIA, AMD, TPU, and Trainium backends from one codebase
- Explicit platform API (`zml.Platform`) for deterministic placement, memory, and I/O management
- Pinned memory allocation and overlapped copy primitives for DMA‑level data transfer speeds
- Zero‑copy integration with standard I/O writers, enabling SSD‑rate model loading and GPUDirect‑like performance
- Built‑in userland virtual file system supporting local files, HTTP, S3, and Hugging Face repositories without intermediate downloads
- Minimal, sandboxed runtime binaries that embed only required accelerator libraries, reducing footprint and startup overhead
- Composable design that separates compilation, execution, and resource handling, avoiding hidden state and implicit behavior