> For the complete documentation index, see [llms.txt](https://research.ambitiousimpact.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://research.ambitiousimpact.com/ambitious-impact-eval/evaluative-criteria.md). # Evaluative criteria The team recommends ideas based on four main types of evaluative criteria. Although different research processes and methodologies may be used to source and narrow down different non-profit intervention ideas, our decision-makers evaluate all potential recommendations based on these criteria. 1. Counterfactual Impact 2. Tractability or Potential for Success 3. Neglectedness 4. Relevance Our evaluative criteria are forward-looking and contextual. We are trying to forecast the potential success of a non-profit idea within the context of AIM’s Charity Entrepreneurship Incubation Program (CEIP). The evaluative criteria are flexible because we move across different topic areas. For instance, what “highly evidenced” means in medical device development differs greatly from what it may mean for an animal welfare non-profit intervention idea. ## The criteria in more detail This section describes the criteria in detail and guides understanding of what the research team aims to achieve when evaluating ideas. In each round, the team operationalizes the criteria into a rubric for scoring. ### Counterfactual Impact * Cost-effectiveness: To be recommended, a non-profit intervention must be expected to beat a certain bar for impact. For Global Health and Development ideas, the historical bar for cost-effectiveness has mostly been below USD 60 per consumption doubling or USD 150 per Disability Adjusted Life Year (DALY) – the more cost-effective below that bar, the better. For Animal Welfare ideas, the bar is over 12 Suffering Adjusted Days (SADs) averted per USD. Depending on the problem area, we are likely to adjust the bar, and we review said bar ahead of each research round. * Maximal impact: We aim to maximize the expected value of the potential non-profits we can start; this involves balancing cost-effectiveness considerations with others, such as the potential scale of the intervention (e.g., the number of people a non-profit could reach to treat a certain illness). ### Tractability or Potential for Success * Well-evidenced: A non-profit intervention idea must have robust evidence to support it. We adjust what robust evidence means based on reasonable expectations for a certain field (e.g., whether one could reasonably expect to find randomized trials on a subject). * High chance of success: We attempt to forecast the likelihood that a non-profit intervention idea would achieve its aims based on various inputs. * Strong Theory of Change (ToC): We favor robust ToCs with few speculative assumptions and a rational connection from inputs to impact. ### Neglectedness * Clear additionality: We recommend non-profit ideas when we have identified a clear role and space they would fill within their ecosystem and, most importantly, would lead to counterfactual impact by not duplicating efforts. * Ecosystem funding considerations: We attempt to avoid situations where, by introducing new non-profits into space, other organizations that we view to be doing good work would miss out on funding. ### Relevance By relevant, we mean an idea considered a good fit with the CEIP. This includes, but is not limited to, the following factors: * Risk (reputational, security, risk of harm to service users, etc.): Our risk appetite varies across and within rounds depending on the portfolio and other strategic priorities. * As a default, we are very risk averse when risking harm against service users, meaning we will likely decline to recommend non-profits with ambiguity about potential harm to service users. * Our appetite for reputational harm varies and is considered more on a case-by-case basis. We often are less risk-averse when working on issues that affect non-human animals. * We consider the security risk of working in volatile or unsafe environments and prefer non-profit intervention ideas offering safer work environments. Ultimately, though, where to work lies with the incubated non-profit leaders, not us. * Incubatee profile fit: We only cause a positive impact in the world if participants in the CEIP choose and deploy our non-profit intervention ideas. We therefore consult with other AIM departments to ensure our choices reflect both: * Ideas likely to be picked up by participants: We avoid recommending ideas that will not attract participants. This is more of a disqualifier than a factor that increases the chances of recommendation (i.e., an idea is unlikely because it will be attractive to incubates, all else equal). * Ideas that fit the profile of our typical incubatee: Our typical incubatee is problem-agnostic and focuses on either human or animal welfare. They are often generalists with no topic-specific skills. We look to increase the odds that the non-profit organization will have a good leader by choosing ideas suited to that type of profile. * Portfolio within the round: In any given research round, we aim to diversify the types of ideas we recommend. This means that all else is equal; we prefer an idea with a different implementation model or working on different problems rather than the same. * Portfolio across AIM incubation: More widely, we also aim to diversify the types of bets we make across rounds, ranging back to our first incubated non-profits. Keeping in mind the difficulty of attributing the success or failure of certain non-profits to the research we do (rather than luck, co-founder quality, funding availability, and more), we try to learn from prior experience and adapt our thinking about which types of implementation models, countries of action, and other factors seem to be aligned with reaching a successful outcome.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://research.ambitiousimpact.com/ambitious-impact-eval/evaluative-criteria.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.