The startup provides a platform for businesses to deploy compressed large language models (LLMs) directly on user devices, significantly reducing AI inference costs and latency while ensuring offline accessibility. Their tools facilitate secure model compression and thorough performance testing across various hardware, enabling seamless integration into existing applications.
Funding
Funding not disclosed
Founders
Product
Problem
Cloud-based AI inference can be expensive and slow, especially for real-time applications, and requires a constant internet connection. Deploying large language models (LLMs) on user devices presents challenges related to model size, computational resources, and security.
Solution
Dereka provides a platform for compressing and deploying LLMs directly on user devices, enabling low-latency, offline AI inference while minimizing costs. Their technology reduces model size by up to 20x without significantly sacrificing performance, allowing integration into applications with limited resources. The platform offers tools for thorough performance testing across various hardware configurations and secure integration into existing codebases, ensuring reliable and safe deployment. By enabling local AI processing, Dereka addresses concerns related to data privacy and eliminates the need for constant cloud connectivity.
Target Audience
The primary target audience includes businesses and developers seeking to deploy AI applications on user devices, particularly those requiring offline functionality, real-time processing, or enhanced data security.
Features
- One-click model compression that reduces LLM size to between 30MB and 500MB
- Performance testing tools to evaluate model accuracy and speed on different devices
- Flexible integration options, allowing deployment on a limited number of devices initially
- Support for offline AI application functionality, enabling access in areas without internet
- Real-time AI translation and inference capabilities with reduced latency
- Enhanced security for AI applications handling sensitive user data