Questions
Tournaments
Services
News
Questions
Tournaments
Questions
Questions
More
Log in
Sign Up
a
/
文
Log in
Sign Up
19
comments
25
forecasters
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
community
30.9
result
29.38
Share
Forecast Timeline
1d
1w
2m
all
Comments
Timeline
Question Info
Resolution Criteria
Background Info
Follow
embed
Authors:
romeodean
Opened:
Mar 24, 2024
Closes:
Dec 31, 2024
Resolved:
Jan 28, 2025
Spot Scoring Time:
Mar 27, 2024
AI Benchmarks
AI Technical Benchmarks
Artificial Intelligence
🏆 2024 Leaderboard
Similar Questions
What will be the best score by an AI on the full Humanity's Last Exam (HLE) before 2026?
51.9%
(40.1 - 62.5)
51.9%
(40.1 - 62.5)
54
forecasters
When will an AI achieve an score of 1.5 or higher in the RE-bench at any time budget between 8h and 32h?
19 Aug 2027
(22 Nov 2026 - 10 Aug 2028)
19 Aug 2027
(22 Nov 2026 - 10 Aug 2028)
46
forecasters
What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2030?
96.7%
(92.3 - 98.7)
96.7%
(92.3 - 98.7)
11
forecasters
Show More Questions