Vals AI
Vals AI is an independent evaluator of artificial intelligence systems, building benchmarks that measure whether models can perform real professional work in law, finance, engineering, and medicine. The company uses privately held test sets to prevent training to the test and open-sources its evaluation infrastructure, including Valkyrie, a distributed system for running agentic benchmarks at scale.
- Artificial Intelligence
- Data & Analytics
- Developer Tools
- Software Only
Funding
Founders
Founder details are not available yet.
Product
Problem
Rapid progress in AI has outpaced the community's ability to construct new evaluation benchmarks, and test sets released openly are quickly absorbed into pre-training corpora, invalidating results. New benchmarks are often produced or run by the same companies that build the models, leading to cherry-picked examples and evaluation regimens tuned to score well. Trillions have been invested in generating intelligence, but comparatively little in measuring it, leaving labs, enterprises, and governments without credible independent assessment.
Solution
Vals AI provides independent evaluation of AI systems through privately held test sets and reproducible infrastructure. The company builds benchmarks that measure whether models can handle real professional work in law, finance, engineering, and medicine, partnering with reference institutions to develop representative task taxonomies and automated scoring methodologies. Scores are based on privately held test sets that prevent training to the test, preserving the integrity and signal of results. Vals AI also open-sources its evaluation infrastructure, including Valkyrie, a distributed system for running agentic benchmarks, and a free model library API, while producing public-good benchmarks like Public Benefits Bench for SNAP questions.
Target Audience
Primary users are AI labs seeking credible demonstration of model progress, enterprises evaluating AI adoption and ROI, and governments needing expertise to measure frontier capabilities and cyber risk.
Features
- Privately held test sets that prevent training to the test and preserve benchmark integrity
- Valkyrie, an open-source distributed system for running agentic benchmarks reproducibly at scale
- Model library, a free standard API to call models with for evaluation purposes
- Public leaderboards tracking accuracy, cost, and latency across dozens of frontier models on benchmarks like LegalBench and the Vals Index
- Benchmarks developed with reference institutions in law, finance, and government, including the Public Benefits Bench built with the Center for Civic Futures and Code for America
- Automated scoring methodologies for generated work product quality in professional domains
- Industry-specific indices such as the Vals Legal Index, which weights practice areas by their share of U.S. law-office revenue