Conditional Trees for AI Risk: A New Question Series and Report
7
This month, the [Forecasting Research Institute](https://forecastingresearch.org/) (FRI), released a new report titled “[Conditional Trees: A Method for Generating Informative Questions about Complex Topics](unsafe:FRI),” which discusses the results of a case study that used a systematic approach (the “conditional trees” method) to generate informative questions about AI risk. While the questions elicited in the case study focus on potential risks from advanced AI, the processes that FRI presents can be used to generate valuable questions across fields where forecasting can help decision-makers navigate complex, long-term uncertainties. *tl;dr—In FRI’s study, the conditional trees approach produced questions with higher "value of information" [1]—meaning they caused greater updates (in expectation) to experts’ beliefs about long-term AI risk—compared to popular AI questions on existing forecasting platforms.*[2] **You can forecast on the questions from the case study in the new** [**Conditional Trees: AI Risk**](https://www.metaculus.com/project/3508/) **question series.** ## Methods FRI researchers interviewed 24 participants, including 21 AI and existential risk experts and three highly skilled generalist forecasters (“superforecasters”). FRI first asked participants to provide their personal forecast of the probability of AI-related extinction by 2100 (the “ultimate question” for this exercise [3]). Participants were then asked to identify plausible [4] indicator events that would significantly shift their estimates of the probability of the ultimate question. Following the interviews, FRI converted these indicators into 75 objectively resolvable forecasting questions (the “AICT” questions). FRI then asked a group of eight superforecasters to provide forecasts on each of these 75 questions, and forecasts on how their beliefs about AI risk would update if each of these questions resolved positively or negatively. FRI quantitatively ranked the resulting indicators by Value of Information (VOI), a measure of how much each indicator caused superforecasters to update their beliefs about the ultimate question. To evaluate the informativeness of the conditional trees method relative to widely discussed indicators, FRI assessed a subset of these questions using a standardized version of VOI, comparing them to popular AI questions on existing forecasting platforms (the “status quo” questions). The status quo questions were selected from two popular forecasting platforms (Metaculus and Manifold) by identifying the highest-engagement AI questions (by number of unique forecasters). FRI presents the results of this comparison in order to provide a case study of a beginning-to-end process for producing quantitatively informative indicators about complex topics. ## Results The case study resulted in four key findings: 1 - Preliminary evidence suggests that the conditional trees method of generating forecasting questions can result in questions that perform better on VOI metrics than popular questions on existing forecasting platforms. Descriptions of the questions shown below can be found in an [appendix](https://forecastingresearch.org/s/AIConditionalTrees.pdf#page=65) to the report.  2 - The conditional trees method produced questions with a markedly different distribution of topic areas compared to those on existing forecasting platforms. Notably, the conditional trees approach led to a greater proportion of questions focused on AI alignment and potential AI harms, reflecting that certain expert priorities may be underrepresented in existing forecasting efforts.  3 - In FRI’s limited sample, experts tended to find questions related to alignment and concrete harms caused by AI to be the most informative. 4 - The conditional trees method as implemented in this case study is particularly labor intensive. FRI expects that the most broadly useful versions of the process would take the underlying principles and 1) apply them to shorter interviews, with smaller numbers of forecasting questions to operationalize, 2) leverage LLMs for elicitation and synthesis, and/or 3) utilize crowdsourcing at the question generation and filtering steps. ## Now It’s Your Turn FRI is excited to share some of the highest-VOI questions from its case study with the Metaculus community. You can forecast on them in the [Conditional Trees: AI Risk](https://www.metaculus.com/project/3508/) question series. --- [1] “Value of information” (VOI) is a metric developed by FRI to measure how much knowing the answer to a binary question about an indicator event would change someone's beliefs about an important outcome. VOI is based on the forecasters’ expected update in their belief about the important outcome, not on how much a participant would update if an event happened. That is, it takes into account how likely the forecaster believes an event is to occur. For more information, see the report section on VOI. [2] This claim is not a criticism of forecasting platforms, which are generally not optimizing for questions that AI experts find informative about long-term AI risk. Rather, it highlights that a more systematic approach to eliciting expert beliefs (such as the conditional trees method) could provide researchers and policymakers with better signals regarding AI risk than crowdsourced questions currently available on popular platforms. [3] More specifically, the ultimate question was “Will the global human population fall below 5,000 individuals at any time before 2100, with AI being a proximate cause of such reduction?”. [4] “Plausible” means that the forecaster deemed the indicator event to be at least 10% likely to occur. This 10% probability was not necessarily an unconditional probability but may have been conditional on a previous node in the conditional tree.