Measuring AI Ability to Complete Long Tasks
Kwa et al., 2025, Measuring AI Ability to Complete Long Tasks (arXiv preprint arXiv:2503.14499)
Subscriber deck — free during early access
1 cards in “5 Extrapolation” · whole deck
5 Extrapolation
- What does extrapolating the trend predict, and with what caveats?AI agents with a one-month (167 working hours) horizon on such software tasks around 2028–2031, sooner if the faster 2024–2025 trend holds. But real tasks often lack automatic scoring, involve other agents, limited resources and changing environments, and agency training, compute limits or automated AI research could change the trend.
- Kwa et al., 2025, Measuring AI Ability to Complete Long Tasks (arXiv preprint arXiv:2503.14499), §5
Source
- Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles, Seraphina Nix, Tao Lin, Chris Painter, Neev Parikh, David Rein, Lucas Jun Koba Sato, Hjalmar Wijk, Daniel M. Ziegler, Elizabeth Barnes and Lawrence Chan. 2025. Measuring AI Ability to Complete Long Tasks. In arXiv preprint arXiv:2503.14499. https://arxiv.org/abs/2503.14499