(10/4 Update) AI Forecasting Benchmark Series — 10/8 Q4 Launch and More
8
*In this post: important information on the upcoming AI Forecasting Benchmark tournament for Q4* ## **October 4th Update** 1. We previously announced that the API was forward-compatible and that users could begin testing the new GET request format. Unfortunately, this was in error, and users will be able to begin using the new format starting Monday, October 7th after the site rewrite goes live. We very much apologize to any who already attempted to update to the new format. (Note: POST requests in the new API will work the same way after the rewrite. There is no need for users to make updates to their existing POST request structure.) 2. The first week of the Forecasting Benchmark Series will consist of unscored warm-up questions. This will give participants space to test their bots before scoring begins on October 15th. 3. We will soon make updated forecasting bot templates available, for any participants who would like to build on these. ## Initial September 30th Post **Mark your calendar apps:** We will launch the next $30,000 phase of the [AI Forecasting Benchmark Series](https://www.metaculus.com/aib/) on October 8th! Further, for everyone competing in Q3 — or curious about current AI forecasting capabilities — we will also share the results of the Q3 competition on the 8th. Those who have already [read our methodology](https://www.metaculus.com/notebooks/28552/comparing-forecasting-track-records-for-ai-benchmarking-and-beyond/) or [x.com thread](https://x.com/metaculus/status/1839069315422646456) know the broad outlines of Bots vs. Metaculus Community vs. Pros, but on 10/8 we'll share the winners, the prize payouts, and some of what we learned running Q3. More actionable in the near term: We are updating our API as part of our in-process site rewrite, which itself accompanies our going [open source](https://www.metaculus.com/notebooks/28595/updates-to-metaculus-api/). The only changes that should affect API users are to the Question endpoint. Whether you are an API user or only use the API in the context of your forecasting bot, you'll want to review [this doc](https://www.metaculus.com/notebooks/28595/updates-to-metaculus-api/#). (Note: See the top of this post for an important update. The update actually goes live October 7th.) Next, for anyone who is on the fence about Q4, we will share updated and improved bot templates — as well as instructional videos — that will help you get started even more easily. If you began building a bot in Q3 but didn't finish it, we strongly encourage you to participate in Q4. And keep in mind: Just as with Q3, we will provide API credits for Anthropic and OpenAI models — as well as services like Perplexity and AskNews — so that you won't be out of pocket on your bot expenses. Just describe how you plan to provide your bot and what models / services you'll plan to use. Finally, we want to note that Q4 will begin with binary questions — but, unlike Q3, we will transition in the 2nd half of October to including additional question types, such as numerical questions. To help you and your bot prepare for this, we will share code you can copy and practice questions you can test your bot on at least a week in advance of new questions. We'll look forward to sharing more updates as we approach the Q4 launch. Feel free to share questions and feedback in the comments!