Researchers Introduce AhaBench to Test How AI Agents Learn from Experience

A new research preprint on arXiv has introduced AhaBench, an evaluation suite designed to test whether language model agents genuinely learn from prior experience across long-horizon workflows. Rather than simply evaluating isolated trajectories, the benchmark measures whether an agent's performance improves when visible support is removed, delayed, or modified.

The benchmark consists of three core environments: Aha-Puzzle for no-hint exploration after puzzle-solving, Aha-Euler for mathematical problem transfer, and Aha-Vending, a simulator testing whether an agent can maintain operational profitability during incidents. The scorecard tracks an Initial Score, a Post-Experience Score, and the net Learning Lift.

Across an eight-model evaluation panel, Claude Opus 4.6 achieved the highest aggregate Post-Experience Score at 64.3 alongside a Learning Lift of +25.8, closely followed by Gemini 3.1 Pro at 63.4. The researchers have open-sourced the benchmark tasks, validators, and simulation code for the AI research community.