Build Your Own Forecasting Bot in 30 Minutes With Our Tutorial Video
8
### February 21 Update Have you been hesitating to build a forecasting bot or hit obstacles in previous attempts? We created a [**step-by-step video guide**](https://www.loom.com/share/fc3c1a643b984a15b510647d8f760685) that takes you from template to working bot in 30 minutes. Remember: 1. Success is achievable: Our template bot has reached the top 10 for two consecutive quarters. A basic bot using o1-preview and AskNews ranked 6th last quarter. With thoughtful adjustments, you could do even better. 2. Time isn't a barrier: Initial setup takes just 30 minutes. Last quarter's winner spent 15-40 hours total and earned \$9,500 (\$240/hour) — that's only 1-3 hours per week. 3. Resources are free: Thanks to OpenAI and Anthropic's donated credits, we're offering free LLM calls through our proxy. Email [support@metaculus.com](mailto:support@metaculus.com) and we'll get you set up. There's still time to compete for \$30,000 in Q1 prizes, and it's never too early to prepare for Q2. ### January 20 Update The AI Forecasting Benchmark for Q1 Is Live! If you haven't yet, register your bot [here](https://www.metaculus.com/aib/). For resources on getting your bot started and for full rules, visit the [tournament page](https://www.metaculus.com/tournament/aibq1/).  *** Two quarters down and two quarters remain for our [**<u>AI Forecasting Benchmark</u>**](https://www.metaculus.com/aib/)**,** which aims to assess how the best bots compare to the best humans on real-world forecasting questions. In this post: * Congratulations to the Q4 winners! Together, they take home \$30,000 in prizes. * [<u>Q1 warm-up questions are open</u>](https://www.metaculus.com/tournament/aibq1/), featuring new question types and important updates to the tournament structure. * Q1 scored questions launch January 20th! ### First, the winners of Q4 Congratulations to the top-performing bots from [<u>Q4</u>](https://www.metaculus.com/tournament/aibq4/)! 1st place: 🥇 pgodzinai - \$9,658 2nd place: 🥈 MWG - \$4,477 3rd place: 🥉 GreeneiBot2 - \$3,930 4th place: 🏆 manticAI - \$3,410 5th place: 🏆 histerio - \$3,312 A special mention to consistent competitors **MWG** and **histerio**, who placed 2nd and 3rd respectively in Q3. And though it claims no prize money, Metaculus’s recency-weighted CP *would* have placed 2nd in this contest, in confirmation of just how difficult it is to overcome the aggregate. Winners: You will receive an email with next steps on prize distribution in a couple days. Please be ready to provide your bot descriptions. **We will later release an in-depth analysis of the bots’ performance overall vs. humans and what we learned in Q4.** ### What you need to know to [<u>forecast in Q1</u>](https://www.metaculus.com/tournament/aibq1/) There will be: * **A mix of multiple-choice, numeric, and binary questions** comparable to those found on Metaculus. * **Randomized question timing** to reinforce the principle of “no human in the loop.” * Questions will open at random times. * Some will remain open for only 1-2 hours. * Up to 10 questions may launch simultaneously. * **A warm up period**: Unscored questions are open now until **January 19th** to give bot makers time to refine their creations. Here are resources to get you started in Q1 — full details about the tournament, as well as the warm-up questions, can be found on the [<u>tournament page</u>](https://www.metaculus.com/tournament/aibq1/). * An enhanced bot template with scheduling functionality, which you can access[ <u>here</u>](https://github.com/Metaculus/metac-bot-template). * [<u>Instructional resources</u>](https://www.notion.so/metaculus/Instructions-Resources-for-Bot-Building-8995fc6f18034f52af8d9b084936a2b5) for bot creation — though note that we are winding down the Google Colab template bot. If you plan to use a template, build from the enhanced bot template linked above. ### For returning bot makers Participated in Q3 or Q4? Here’s what you need to know: * You will need to request new credits * The proxy location has been updated. (See the tournament page for details.) ### Warm-up questions Unscored practice questions are live [<u>here</u>](https://www.metaculus.com/tournament/aibq1/). Short-lived questions will open each hour until scored questions launch on January 20th so you can prepare your bot for the new contest structure. ### Analysis from previous rounds of the contest [<u>How did bots perform in Q3?</u>](https://www.metaculus.com/notebooks/28784/aibq3results/) [<u>We tested a bot built on OpenAI’s o1-preview model. How did it do?</u>](https://www.metaculus.com/notebooks/30889/self-consistency-of-gpt4o-vs-o1-preview/) ### Why a Forecasting Benchmark? Forecasting benchmarks measure key AI capabilities like strategic thinking and world-modeling. Metaculus questions often require complex reasoning and sound judgment, making it difficult to game the system. While AI forecasting accuracy still lags behind humans, the gap is closing, and tracking this progress is crucial. In addition to accuracy, we evaluate metrics like calibration and logical consistency, offering a comprehensive view of AI performance. This series invites you to create your own forecasting bot, compete for \$120,000 in prizes, and contribute to understanding AI’s evolving capabilities. Scroll ahead to learn how to get started. #### *Want to discuss bot-building with other competitors? There’s a lively Discord channel just for that. Join it*[ *<u>here</u>*](https://discord.gg/NJgCC2nDfh)*.*