InferX provides a serverless inference platform that runs any Docker/OCI container or vLLM image via an OpenAI‑compatible API, delivering sub‑2 second cold starts and automatic GPU eviction to eliminate idle costs. It offers fine‑grained GPU slicing with isolated compute, pay‑per‑execution billing, and instant scaling for bursty AI workloads, enabling developers to deploy and update models without vendor lock‑in.
Funding
Funding not disclosed
Founders
Product
Problem
Developers deployinglarge language models or custom AI containers face high idle GPU costs, complex DevOps workflows, and long cold‑start times that hinder rapid experimentation and scaling, especially for bursty or low‑utilization workloads.
Solution
InferX delivers a serverless inference platform that lets users upload any Docker/OCI container or vLLM image and access it through a standard OpenAI‑compatible API. The service creates NVMe snapshots of the model, loads them onto GPUs in under two seconds, and automatically evicts idle GPUs within milliseconds, achieving true scale‑to‑zero operation. Fine‑grained GPU slicing (e.g., 1/8, 1/4) provides isolated compute resources, and the platform scales instantly to handle traffic spikes without requiring warm pools. Billing is per execution, eliminating idle charges and over‑provisioning. The platform requires no proprietary SDKs, enabling drop‑in replacement of existing integrations.
Target Audience
Primary customers are AI developers, data‑science teams, and SaaS or enterprise providers that need on‑demand inference for applications with variable traffic such as chatbots, agents, analytics services, and custom AI pipelines.
Features
- Serverless, pay‑per‑execution pricing with sub‑2 second cold starts and automatic GPU eviction after idle periods
- OpenAI‑compatible REST/gRPC API for seamless integration with existing client code
- Fine‑grained GPU slicing (1/8, 1/4, 1/3, 1/2, full) with VM‑level isolation to prevent cross‑tenant interference
- Support for any Docker/OCI container or vLLM image, avoiding vendor lock‑in
- NVMe snapshot‑based model storage enabling instant GPU provisioning and rapid model updates
- Automatic scaling that provisions fresh GPUs in seconds, handling viral traffic spikes without warm pools
- High GPU utilization (80%+ average) through just‑in‑time scaling and dense deployment