Cumulus Labs provides a serverless GPU inference platform that lets ML teams deploy models with a single API call, delivering a secure HTTPS endpoint and sub‑15‑second cold starts. The service automatically scales GPU replicas across cloud regions and charges only for active compute cycles, eliminating idle costs. For private deployments, Cumulus OS offers the same serverless engine to orchestrate on‑prem GPU clusters.
Funding
Funding not disclosed
Founders
Product
Problem
Machine learning teams often face high latency and cost when provisioning GPU resources for inference, especially due to long cold‑start times and paying for idle compute. Managing GPU scaling, failover, and region selection adds operational overhead that slows product rollout.
Solution
Cumulus Labs delivers a serverless GPU inference platform that eliminates infrastructure management. Users deploy any model with a single function call, receiving a unique model ID and a ready‑to‑use HTTPS endpoint. The service achieves sub‑15‑second cold starts by leveraging memory snapshotting and torch.compile optimizations, while automatically scaling replicas across cloud regions based on request volume. Billing is metered per GPU compute cycle, so costs drop to zero when the endpoint is idle. For organizations that require on‑premise control, Cumulus OS provides private GPU cluster orchestration powered by the same serverless engine.
Target Audience
The primary customers are ML engineers, data‑science teams, and AI product developers who need low‑latency, cost‑effective inference at scale, as well as enterprises seeking a private, serverless GPU cluster for on‑premise workloads.
Features
- One‑line SDK/API that provisions a containerized GPU function and returns a model_id and inference endpoint
- Cold‑start latency as low as 12.5 seconds using memory snapshot and torch.compile acceleration
- Autoscaling engine that adds or removes GPU replicas in real time across multiple cloud regions, with built‑in failover and GPU selection logic
- Pay‑per‑compute pricing model that charges only for active GPU cycles, eliminating idle‑time charges
- Secure, TLS‑encrypted endpoints with role‑based access controls for multi‑tenant usage
- Cumulus OS: on‑premise GPU cluster manager that mirrors the serverless workflow for private clouds
- Language‑agnostic invocation (Python, REST, Node, etc.) and integration hooks for CI/CD pipelines
- Built‑in monitoring dashboard showing replica count, latency metrics, and cost analytics