TokenJam is an open‑source, locally‑run observability platform that collects telemetry from major LLM agent runtimes, frameworks, and cloud providers. It visualizes usage and applies cost‑optimization analyzers—such as model downgrades and prompt‑bloat trimming—through a single CLI install, helping developers reduce token spend by up to 86% on flagged sessions.
Funding
Funding not disclosed
Founders
Product
Problem
AI agents and LLM-powered applications generate large volumes of token usage, prompt bloat, and redundant processing, leading to unexpectedly high cloud bills. Existing observability tools often require cloud services, external sign‑ups, or lack actionable cost‑reduction guidance.
Solution
TokenJam provides an open‑source, locally run observability and cost‑optimization layer for AI agents. It ingests telemetry from major LLM providers, agent frameworks, and OpenTelemetry sources, then runs five analyzers—Downsize, Trim, Cache, Script, and Reuse—to surface concrete savings opportunities. Downsize identifies sessions that could use cheaper models within the same family, while Trim flags low‑significance prompt sections using a local LLMLingua‑2 classifier. Cache highlights stable prompt prefixes and recommends provider‑native caching breakpoints. Script and Reuse detect deterministic tool‑call patterns that can be replaced by plain scripts or reusable templates. All analysis runs on the user’s machine, with no data egress, and results are presented via a CLI, local web UI, and exportable configuration files for immediate implementation.
Target Audience
Primary users are developers and engineering teams building AI agents or LLM‑driven applications who need detailed cost visibility and actionable optimization without sending telemetry to the cloud.
Features
- CLI‑driven optimizers that analyze up to millions of spans, token counts, and tool calls across a 30‑day window
- Downsize analyzer suggests model downgrades with example sessions, achieving up to 86 % cost reduction on flagged calls
- Trim analyzer uses a local LLMLingua‑2 model (~280 MB) to highlight and remove prompt bloat without external API calls
- Cache analyzer reports current caching ratios and recommends cache_control breakpoints for provider‑native caching, saving 30–60 % on repeat prefixes
- Script analyzer clusters deterministic tool‑call sequences to replace costly agent loops with lightweight Python scripts
- Reuse analyzer extracts recurring planning skeletons, exports them as reusable templates, and quantifies saved regeneration costs
- Full local observability stack includes real‑time cost tracking, budget alerts, drift detection, and sensitive‑action notifications
- Supports ingestion from 20+ integration points (Anthropic, OpenAI, Gemini, Bedrock, LangChain, AutoGen, OTLP, Langfuse, Helicone, etc.) and custom spans