Question

What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?

7 comments7

Total Forecasters21

Community Prediction38

Make a Prediction

My Prediction

community

lower 25%

median

upper 75%

...

30.98

38.04

46.55

Authors:

romeodean

Opened:

Closes:

Scheduled resolution:

AI Technical Benchmarks

AI Benchmarks

Artificial Intelligence

Performance analysis

Computer science

Software engineering

What will be the best score on the WebArena benchmark before 2025?

When will models hit 97.5% on SWE-Bench (Verified Version)

When will models hit 90% on SWE-Bench (Verified Version)

2025-10-30

Comments

? comments

Authors:

romeodean

Opened:

Closes:

Scheduled resolution:

AI Technical Benchmarks

AI Benchmarks

Artificial Intelligence

Performance analysis

Computer science

Software engineering

Notion And Stripe Trust This Year-Old Startup To Make Sure Their AI Tools Work

Forbes•Oct 11, 2024

UBS sees S&P 500 soaring to 6,400 by the end of 2025

CNBC•Oct 15, 2024

Early automates code testing for developers

TechCrunch•Oct 15, 2024

Learn more about Metaculus NewsMatch

What will be the best score on the WebArena benchmark before 2025?

When will models hit 97.5% on SWE-Bench (Verified Version)

When will models hit 90% on SWE-Bench (Verified Version)

2025-10-30