Outerport provides a caching middleware that enables efficient task switching among multiple AI models on a single GPU, significantly reducing cold start times and improving model loading speeds. By optimizing GPU utilization, Outerport helps companies save up to 40% on GPU costs while maintaining high performance in multi-model workflows.
Funding
$500K raised to dateRaised to date based on public sources. This may differ from the amount the company actually raised and is based only on what is publicly available on the internet.
Founders
Product
Problem
In multi-model AI workflows, frequent model switching leads to inefficient GPU utilization due to cold starts and repeated loading of large models. This results in increased latency, higher operational costs, and underutilization of expensive GPU resources.
Solution
Outerport provides a caching middleware solution that optimizes GPU utilization by enabling efficient task switching among multiple AI models. The middleware caches AI models and tensors, significantly reducing cold start times and accelerating model loading speeds. By parallel loading and page locking, Outerport ensures models are ready to go, facilitating hot-swapping and asynchronous checkpointing. This approach allows for deploying model updates without downtime and amortizing auto-scaling costs, ultimately maximizing GPU utilization and reducing infrastructure expenses.
Target Audience
Outerport targets companies and research institutions that deploy multiple AI models in production, including those involved in agent builders, ComfyUI, and other multi-model workflows.
Features
- Caching middleware designed for AI models and tensors
- Parallel loading and page locking for optimized model loading
- Hot-swapping of models from the cache
- Asynchronous checkpointing for model and tensor movement
- Deployment of model weight updates without container updates or downtime
- Centralized model upload with access policies, audit logs, and quantization/compression policies
- Rust-based daemon process for resilience and model management
- Self-hosted software deployable on any cloud or on-premises infrastructure