Question

If and when this graph is extended to 10^14 parameter models trained on 10^14 elapsed tokens of similar-quality data, will the 10^14 parameter learning curve have slowed down substantially?

22 comments22

Total Forecasters29

Community Prediction

30%

(29% - 50%)

Make a Prediction

50%

community: 30%

No key factors yetAdd some that might influence this forecast.

Add key factor

Authors:

kokotajlod

Opened:

May 18, 2021

Closes:

Feb 23, 2037

Scheduled resolution:

Jan 1, 2050

Spot Scoring Time:

May 20, 2021

AI Training and Compute

AI Technical Benchmarks

Computing and Math

Artificial Intelligence

How many billions of parameters will the largest machine learning model trained before 2030 have?

1.44M

Will a dense machine learning model with at least 100 trillion parameters be trained before 2026?

28%

Will high-impact research on reducing the sample complexity of Large Language Model pretraining be forthcoming before 2026?

60%