> For the complete documentation index, see [llms.txt](https://research.ambitiousimpact.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://research.ambitiousimpact.com/ambitious-impact-eval/cost-effectiveness-modelling/what-benefits-will-the-intervention-generate.md).

# What benefits will the intervention generate?

* Outcome modelling is difficult.
* The core challenge is to try to model effects while accounting for counterfactuals and base rates.
  * Despite how tough that is, **you always have to do this**.
* Generally speaking, the steps to outcome modelling are to:
  1. Establish what would happen without the intervention (say, *counterfactual*)
  2. Establish what would happen with the program (say, *effects*)

$$ \text{Intervention benefits}= Effects - Counterfactual $$

## Accounting for benefits

### Top-down approach

* Models benefits at the population level
* Useful for whole-population approaches, e.g., public policy changes
* E.g., if we reliably know an intervention reduces all-cause mortality by 5% across the whole population, we can simply model those effects at a population level.

### Bottom-up approach

* Models benefits at individual level and adjusts for individuals reached
* Most commonly used
* Best estimation approach depends on data used

### Further guidance

* Figuring out effect sizes from studies is relatively simple, but also frequently requires adjustment to levels of treatment

## Accounting for possible reach

*This advice largely focuses on modelling for forecasts - i.e., future reach. Accounting for existing reach is more straightforward if you have good monitoring and evaluation.*

* Modelling potential reach for a future intervention is challenging because it is usually educated guesswork.
* If you find that a model is very sensitive to reach and your inputs here are just a bit better than a guess, report results at various levels.
* Consider multiple estimation approaches, for instance
  * **t**op-down (”market”-based),
  * bottom-up (operational capacity-based), and
  * reference class thinking (comparative)
* In most complete models you should probably consider ramp-up period, steady-state, and potential decline. For the most, though, part this applies to a very comprehensive dynamic model.
* Reach typically follows an S-curve adoption pattern, not linear growth

### Top down approach - TAM / SAM / SOM

*Technically TAM/SAM/SOM are market sizing tools for the private sector but I am happy to butcher them for our purposes.*

```mermaid
graph LR
  A[**Total Addressable Market**<br>*Every being that could theoretically benefit*]
  B[**Serviceable Available Market**<br>*Filters for geography, demography, access*]
  C[**Serviceable Obtainable Market**<br>Filters for capacity constraints, market share assumptions, etc.]
  A---->B
  B---->C
 
 D[1m total Gronk production in Chonkiland across equally sized 20 farms]
 E[10% farms are very remote, 10% are part of EvilCorp<br>*16 available*]
 F[Assume we can cover 20% of serviceable market<br>*3 farms = 150k Gronks affected*]
 
 D--->E
 E--->F
 
```

#### **Total Addressable Market (TAM)**

* Everyone who could theoretically benefit
* Example: All children under 5 in Country X with condition Y
* Source: Burden of disease data (GBD, DHS, national surveys)

#### **Serviceable Available Market (SAM)**

* Subset meeting geographic, demographic, or practical criteria
* Apply filters:
  * Geographic scope (regions where charity will operate)
  * Demographic targeting (e.g., only rural populations)
  * Access criteria (e.g., must live within 50km of health facility)
* Example: Children under 5 with condition Y in selected districts of Country X

#### **Serviceable Obtainable Market (SOM)**

* Realistic capture given competitive landscape and barriers
* Consider:
  * Existing service provision by government/other NGOs
  * Market share assumptions (typically 5-20% for new entrants in first 3-5 years)
  * Adoption barriers (awareness, trust, cost, cultural factors)
  * Capacity constraints of the new charity
* Example: 10% of children under 5 with condition Y in selected districts, accounting for existing government services

### Bottom-up approach

1. What is the unit of delivery (e.g., farm, individual, hospital, etc.)
2. Estimate, based on capacity, the amount of units reached for the time period.
3. Adjust based on execution quality and other reach factors (conversion rates, etc.)

### Reference Class Forecasting

Use comparable organizations as benchmarks:

1. **Identify reference class**
2. **Gather reach data**
3. **Adjust for differences**
   * More/less funding than reference class
   * Stronger/weaker ecosystem support
   * More/less competitive landscape
   * Use adjustment factors
4. **Apply inside view corrections**
   * Start with reference class median
   * Only adjust upward with strong evidence
   * Document all adjustment rationale

## Temporal dynamics for policy implementation

* When modelling policy dynamics - such as an advocacy intervention, you are going to have to simplify things.
* For instance:

| **Year**                             | **What is happening**          | **Cost**                              | **Benefits** |
| ------------------------------------ | ------------------------------ | ------------------------------------- | ------------ |
| 0 - 4                                | Advocacy                       | Costs of advocacy                     | 0            |
| 5                                    | Passage (probability-weighted) | Cost of advocacy                      | 0            |
| 6+                                   | Implementation                 | Costs to government of implementation | 1            |
| Ramp down or avoid costs to advocacy | S-curve ramp up over 3-5 years |                                       | 1            |

## Validation Checks

1. **Per capita comparison**: Does reach/population align with similar programs? (typically 0.1-5% in early years)
2. **Capacity utilization**: Are staff working >100% capacity? (flag if yes)
3. **Market share logic**: Does reaching X% of need seem plausible given actors present?
4. **Funding arithmetic**: Budget ÷ cost per beneficiary = mathematically possible reach
5. Reaching >20% of TAM in first 3 years (unless very small TAM)
6. Growth >100% year-over-year sustained beyond Year 3
7. No acknowledgment of existing service provision
8. Linear growth assumptions (S-curves are more realistic)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://research.ambitiousimpact.com/ambitious-impact-eval/cost-effectiveness-modelling/what-benefits-will-the-intervention-generate.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.