Questions
Tournaments
Services
News
Questions
Tournaments
Questions
Questions
More
Log in
Sign Up
a
/
文
Log in
Sign Up
19
comments
25
forecasters
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
community
30.9
result
29.38
Share
Forecast Timeline
1d
1w
2m
all
Comments
Timeline
Question Info
Resolution Criteria
Background Info
Follow
embed
Authors:
romeodean
Opened:
Mar 24, 2024
Closes:
Dec 31, 2024
Resolved:
Jan 28, 2025
Spot Scoring Time:
Mar 27, 2024
AI Benchmarks
AI Technical Benchmarks
Artificial Intelligence
2024 Leaderboard
Similar Questions
What will be the best score by an AI on the full Humanity's Last Exam (HLE) before 2026?
60.6%
(49.5 - 72.8)
60.6%
(49.5 - 72.8)
47
forecasters
When will an AI achieve an score of 1.5 or higher in the RE-bench at any time budget between 8h and 32h?
25 Dec 2027
(26 Mar 2027 - 15 Dec 2028)
25 Dec 2027
(26 Mar 2027 - 15 Dec 2028)
38
forecasters
What will be the best non-human SAT-style score on the hard subset of the QuALITY dataset by January 1, 2030?
96.7%
(92.3 - 98.7)
96.7%
(92.3 - 98.7)
11
forecasters
Show More Questions