Squarediff provides an autonomous experimentation platform for AI agent teams, automatically generating, deploying, and evaluating hundreds of agent variants in parallel. The system creates hypotheses from performance metrics, execution traces, and the latest research, then surfaces statistically significant results and exports the top version as a ready‑to‑merge GitHub pull request, integrating with major agent frameworks for seamless adoption.
Funding
Funding not disclosed
Founders
Product
Problem
AI teams struggle to improve agents because testing is slow, manual, and limited to a few hypotheses per week, while advances in prompting and architecture appear faster than they can be incorporated.
Solution
SquareDiff offers an autonomous experimentation platform that generates, codes, deploys, and evaluates hundreds of agent variants in parallel. The system derives hypotheses from evaluation scores, execution traces, and the latest research, then runs them automatically to identify the highest‑performing version. Results are presented with statistical significance, and the winning variant can be exported as a ready‑to‑merge GitHub pull request. The platform integrates with existing agent frameworks, allowing teams to adopt it without rewriting code and to continuously refine evaluation metrics using real‑world business outcomes.
Target Audience
Primary customers are AI product teams and engineering groups that build conversational or autonomous agents and need rapid, data‑driven iteration to stay competitive.
Features
- Automated hypothesis generation using evaluation metrics, trace analysis, and frontier research insights
- Parallel creation, deployment, and scoring of 100+ agent variants to accelerate discovery
- Statistical tracking of accuracy, cost, latency, hallucination, adversarial robustness, and other custom metrics
- One‑click export of the top‑performing variant as a GitHub PR for immediate production rollout
- Multiple experimentation modes: autonomous generation, suggested ideas, curated templates, and manual natural‑language test definition
- Native connectors for major agent frameworks (LangChain, CrewAI, AutoGen, LlamaIndex, Semantic Kernel, Haystack, OpenAI Agents, Vercel AI SDK, DSPy, etc.) with custom integration support