Zapdev-labs
Zapdev-labs builds open-source tools for quantizing and running machine learning models locally on consumer hardware. Its projects include turboquant for model compression, oxidize as a dependency-light C11 inference runtime, and rerunnable benchmark suites that compare performance across backends. The lab reports oxidize and miniforge outperforming stock llama.cpp on its own hardware tests.
- Artificial Intelligence
- Developer Tools
- Software Only
Funding
Founders
Founder details are not available yet.
Product
Problem
Machine learning models are increasingly large and costly to deploy, pushing teams toward cloud dependence and away from local, private inference. Standard quantization and inference stacks often deliver suboptimal performance on consumer hardware, leaving developers without transparent, reproducible ways to measure and improve their deployment efficiency.
Solution
Zapdev-labs provides open-source tools and workflows for quantizing and benchmarking machine learning models, with a focus on local inference optimization. The platform offers an "oxidize" rewrite that strips dependencies down to a C11 core with explicit CPU and CUDA backends, delivering measurable speed gains over stock stacks. Developers can run back-to-back benchmarks on the same model and prompt across different runtimes, producing relative tokens-per-second scores that guide deployment choices. The projects are built in public, with source available on GitHub and releases tracked against an active rewrite.
Target Audience
Developers and ML engineers running open-source language models on local hardware, plus research teams working on quantization, inference optimization, and reproducible benchmarking.
Features
- Turboquant quantization workflow enabling compression, including 3-bit KV cache keys that reduce KV cache size by 6×
- Oxidize: a dependency-light C11 inference runtime with a native CLI and explicit CPU and CUDA backends
- Miniforge and turboforge: local inference stacks built for performance comparison against llama.cpp and Ollama
- Rerunnable benchmark suite that tests each backend on the same GGUF file and prompt, with relative tokens-per-second reporting
- Openagentmail: agent infrastructure for AI agent communication
- 12+ open-source projects hosted publicly on GitHub with build-from-source instructions