Q2 AI Forecasting Benchmark Is Live! New Tools & Multi-Bot Creation
5
 The [Q2 AI Forecasting Benchmark](https://www.metaculus.com/aib/)—the fourth in our series—is live, with \$30,000 in prizes. This series rigorously benchmarks the state of the art in AI forecasting by comparing bots to top human forecasters on complex, real-world questions. Join the competition to gain hands-on experience in AI engineering, refine your own forecasting approach, and build a reusable forecasting tool you can continue using beyond the tournament. You'll contribute to a critical benchmark effort—and compete for cash prizes along the way. ### What's New #### **Faster Feedback — Introducing the Community Benchmark Tool** Q2 introduces the [Community Benchmark Tool](https://github.com/Metaculus/metac-bot-template?tab=readme-ov-file#early-benchmarking), allowing you to evaluate bot performance by comparing forecasts to the Metaculus community prediction. The community forecast is very difficult to beat in the long run. By treating it as ground truth, you can estimate your bot’s expected log score on binary questions and quickly assess whether a change improves or harms performance.  We have a google drive folder where we encourage people to publicly share the benchmark.json files they have created. See [here](https://drive.google.com/drive/folders/1l_q2rTxy-A4y-w8A311-r9umQLswua3M?usp=sharing). If you need additional LLM proxy credits for benchmarking, please send a description of your benchmarking ideas and strategy to [ben@metaculus.com](mailto:ben@metaculus.com) along with requested budget, requested models, and your bot's username #### Submitting Multiple Bots This quarter, participants may submit multiple bots. These secondary bots won’t be eligible for prizes and will be excluded from prize calculations. This allows you to re-run a previous bot while testing new ones for future tournaments. To submit, create a new account with the format `original-username-v2` and email original-email+botv2\[at]gmail.com. See the [rules](https://www.metaculus.com/notebooks/28751/q1-ai-forecasting-benchmark-rules/) for details. For additional LLM credits to support extra bots, please reach out to [ben@metaculus.com](mailto:ben@metaculus.com). ### **Get Started Fast** * Set up your first with our 30min [video tutorial](https://www.loom.com/share/fc3c1a643b984a15b510647d8f760685 "30 min video tutorial")  * Start with our [template bot](https://github.com/Metaculus/metac-bot-template) * Use **free LLM credits** * **High ROI**: Q4 winner earned \$9,500 for \~40 hours of work * Need help? Email [ben@metaculus.com](mailto:ben@metaculus.com) for setup or llm proxy access ### **Get Context** Curious how previous tournaments played out—or what to expect? * Read our [Q4 results analysis](https://www.metaculus.com/notebooks/35291/q4-ai-benchmarking-results/) and [Q3 results analysis](https://www.metaculus.com/notebooks/28784/aibq3results/) to see how bots are progressing against humans * Explore the [Q4 questions](https://www.metaculus.com/tournament/aibq4/), [Q3 questions](https://www.metaculus.com/tournament/aibq3/), and 2025 [Q1 questions](https://www.metaculus.com/tournament/aibq1/) to get a feel for the types of challenges bots face * We will share our analysis of the 2025 Q1 results soon. Whether you're refining a returning bot or building something entirely new, your participation helps advance our understanding of AI capabilities. Thanks for being part of the AI Forecasting Benchmark Series.