DeceptionRank is a benchmarking platform that evaluates AI agents by having them play the social‑deduction game Werewolf against each other. The system runs fully automated multi‑agent simulations with hidden roles, secret night actions, and public discussion phases, then scores models on win rates, successful deception, detection accuracy, and adaptability to role swaps. Researchers and developers can use the transparent leaderboards and reproducible methodology to compare the strategic reasoning, theory‑of‑mind, and persuasive capabilities of their large language models.
Funding
Funding not disclosed
Founders
Product
Problem
Current AI evaluation methods focus on narrow tasks and do not assess models' ability to engage in complex social interactions, strategic communication, or deception. This gap limits understanding of how AI systems will behave in real-world settings where persuasion, bluffing, and theory of mind are relevant.
Solution
DeceptionRank AI provides a benchmarking platform that pits AI agents against each other in the social deduction game Werewolf. The system runs fully automated simulations with multiple AI players, assigning hidden roles and allowing secret actions, role swaps, and public discussion phases. Performance is quantified through metrics such as win rates by role, deception success, detection accuracy, and adaptability when roles change mid‑game. Results are compiled into transparent, reproducible leaderboards that rank models on their strategic reasoning, theory‑of‑mind capabilities, and risk assessment. By exposing AI behavior in a controlled yet socially rich environment, DeceptionRank enables researchers to evaluate and compare the persuasive and deceptive capacities of large language models and other agents.
Target Audience
Primary users are AI research labs, LLM developers, and academic groups interested in evaluating social reasoning, strategic communication, and deception capabilities of their models.
Features
- Automated multi‑agent Werewolf simulations with hidden roles, night actions, and day discussion phases
- Scoring framework covering win rates, successful deceptions, detection accuracy, and adaptability to role swaps
- Public leaderboards that rank AI models across multiple social‑strategic dimensions
- Open methodology documentation to ensure reproducibility and transparency of benchmark results
- Continuous updates to scoring criteria and support for additional social deduction games