> For the complete documentation index, see [llms.txt](https://research.ambitiousimpact.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://research.ambitiousimpact.com/other/our-prioritization-funnel/stage-4-critical-uncertainty-analysis.md). # Stage 4: Critical Uncertainty Analysis {% stepper %} {% step %} ### Identifying critical uncertainties (inc. failure modes) for each idea Aim to list the top 1-6 crucial considerations/key assumptions that need research. Can do some (but not all of) the following to generate crucial considerations: * Look at key considerations (crucial considerations) from the previous stage (10min) * Look at stage 3 CEAs to figure out which numbers are most uncertain and which will make the biggest difference to the final answer. More detailed ToC and intervention description (15 min) * And identify key assumptions (10 min) * Do a COM-B ToC assumptions analysis (30min) Some typical critical considerations include: * Neglectedness (E.g., How many people are working in this space? Is there funding for more actors?) * Tractability (E.g., How likely is it that a charity can achieve X? Have there been any attempts to achieve a policy Y? Have those succeeded? Do people have the time and money to do Z?) * Externalities (E.g., Is it possible that promoting behavioral change Y backfires and causes harm X? Is there a chance that an increase in the consumption of Z displaces other purchases?) * ToC (E.g., Is it possible for a CE-style charity to have the technical skills to deliver this intervention?) * Are there high start-up costs? * Any limiting factors (E.g., Does this burden affect a large enough number of people? Would it be possible to conduct this intervention in a large number of countries?), etc. * Harms matter. Make sure you have considered if there are any potential harms to an idea. * What intuitively feels most likely to kill an idea (5min) * Google evidence against (30min) You can use the critical uncertainties bot to speed up and support your decision-making process - instructions can be found in the stage 4 template. {% endstep %} {% step %} ### Expert conversations If necessary, on a case-by-case basis, reach out to experts to set up interviews * Email 1-4 experts (we are aiming to talk to one expert in the space, but it is unlikely that all will reply so we reach out to more than we plan to interview, if you think it is very likely that an expert will reply to you and that they are the best to speak with on a given topic then you can just reach out to them). There is no need to email all experts – it may be better to talk to them at stage 5 to focus on a specific idea. * Expert interview (1-2 interviews 1h15min-2h45min). Prep questions before the interview. Make sure the questions you ask are tailored to the ToC and that you have plans for what to probe or dig deeper into during the interviews. Although this round is focused on critical uncertainties the interviews should be more general than that, cover a range of topics, ask for other people to talk to and consent to make and use notes, etc. (Treat S4 interviews as if they were S5) * Expert view - Speak to 1-2 experts in the area to get broad information (20min to arrange and write questions + 30min-1hour interview each) * Record interviews – use zoom * Summary writeup – 15min {% endstep %} {% step %} ### Resolving critical uncertainties and failure modes for each idea Choose 2-4 critical uncertainties per idea (min 1, max 6). Make sure they are truly decision-critical. Sometimes it can be tempting to research implementation questions that are not likely to be decision-critical at this stage. Resolve these critical uncertainties in whatever way is best. This can mostly be done through desk research and interviews. Remember: As with all AIM research, you can stop this process early if an idea no longer looks promising after investigating 1 key uncertainty. {% endstep %} {% step %} ### Quality Assurance Another reviewer will read other stage four reports and leave questions and comments on the analysis, particularly highlighting any crucial considerations that haven’t been addressed. {% endstep %} {% step %} ### Decision meetings The round lead (or other) will convene a decision meeting (or more mid-stage meetings) to consider all research done up to this stage and prioritize the short list of ideas. At this stage, there is no specific need to select ideas; the decision meetings should give us a * Sense of the priority ideas to conduct deep reports on. * Cutting ideas that are outright not worth reviewing. {% endstep %} {% endstepper %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://research.ambitiousimpact.com/other/our-prioritization-funnel/stage-4-critical-uncertainty-analysis.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.