Question

What will the state-of-the-art language modelling performance on One Billion Word be on 2023-02-14, in perplexity, amongst models not trained on extra data?

Resolved:20.25

7 comments7

Total Forecasters36

Community Prediction

(19.9 - 21.5)

Make a Prediction

PDF

CDF

	<13	lower 25%	median	upper 75%
community	0.7%	19.93	21	21.5
My Prediction	—	—	—	—

Lower bound	community	My Prediction
<13	0.7%	—
Quartiles
lower 25%	19.93	—
median	21	—
upper 75%	21.5	—

What was the final result?20.25

Community Baseline Score

20.4

Community Peer Score

27.8

Authors:

MetaculusOutlooks

Opened:

Feb 13, 2021

Closes:

Apr 14, 2021

Resolves:

Feb 14, 2023

Spot Scoring Time:

Feb 15, 2021

Forecasting AI Progress

AI Technical Benchmarks

Forecasting AI Progress: Deep Learning Round

Artificial Intelligence

2021-2025 Leaderboard

How many billions of parameters will the largest machine learning model trained before 2030 have?

1.44M

When will a language model be developed that, when tested, yields approximately human-level output?

05 Jun 2024

Will high-impact research on reducing the sample complexity of Large Language Model pretraining be forthcoming before 2026?

60%