19 comments
25 forecasters
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
Forecast Timeline
Authors:
Opened:Mar 24, 2024
Closes:Dec 31, 2024
Resolved:Jan 28, 2025
Spot Scoring Time:Mar 27, 2024
What will be the best score by an AI on the full Humanity's Last Exam (HLE) before 2026?
51.9%
(40.1 - 62.5)
51.9%
(40.1 - 62.5)
54 forecasters
When will an AI achieve an score of 1.5 or higher in the RE-bench at any time budget between 8h and 32h?
29 Aug 2027
(30 Nov 2026 - 08 Sep 2028)
29 Aug 2027
(30 Nov 2026 - 08 Sep 2028)
46 forecasters
What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2030?
96.7%
(92.3 - 98.7)
96.7%
(92.3 - 98.7)
11 forecasters