Why Policy Decisions Depend on Valuation
A causal effect does not, by itself, determine who should receive treatment. A treatment may improve an outcome but also use resources, impose burdens, or have benefits that decision-makers value differently. A policy therefore combines estimated treatment effects with an explicit valuation of outcomes and costs. Sensitivity analysis asks whether the resulting treatment choices and policy value change when those inputs change.
The calculations in this tutorial take conditional treatment effects as given. They do not establish that those effects are identified or estimated precisely; those questions require the causal design and evaluation methods developed earlier. The purpose here is to show how a decision follows from a specified set of effects and valuations, and how that decision responds to changing them.
This formulation uses a linear valuation: an additional unit of the outcome is worth the same amount \(\lambda\) across the range being considered. Costs and burdens are incremental, meaning they are compared with the costs and burdens under no treatment. If a cost is common to both options, it cancels from the comparison. If treatment has several costs, they can be added into \(c(x)\) or \(h(x)\), provided they are expressed in compatible units.
A treatment rule \(d(x)\) equals 1 when a person is assigned treatment and 0 otherwise. For a target population, its incremental policy value relative to treating nobody is
The expectation averages across the characteristics of people in the target population. Thus, policy value is not just the value for those treated: people not treated contribute zero incremental value relative to the no-treatment policy. If the population is represented by groups with proportions \(p_g\), group effects \(\tau_g\), and group costs \(c_g\), then \(V(d)=\sum_g p_gd_g(\lambda\tau_g-c_g-h_g)\).
The Break-Even Rule
When there is no capacity constraint and decisions are made person by person, treating someone increases expected utility exactly when their incremental utility is positive. If treatment is required to exceed a decision hurdle \(r\), the rule becomes \(\Delta(x)>r\). The hurdle is measured in the same utility units as \(\Delta\). A zero hurdle gives the ordinary break-even decision; a positive hurdle can represent a minimum gain required before implementing treatment.
The result follows by considering each person's contribution to policy value. If \(d(x)=1\), the contribution is \(\Delta(x)\); if \(d(x)=0\), it is zero. Choosing the larger of these two contributions at each \(x\) gives \(\max\{\Delta(x),0\}\). No other unconstrained rule can improve the contribution at any \(x\), so no other rule can have greater expected value.
This derivation also shows why costs and outcome valuation affect the selected group. Increasing \(\lambda\) raises the value assigned to a beneficial treatment effect, whereas increasing \(c(x)\) or \(h(x)\) lowers net benefit. For a positive treatment effect, the break-even effect threshold falls as \(\lambda\) rises and rises as costs or burdens rise. These statements concern the decision rule, not whether the underlying causal effect estimate is accurate.
Worked Example: Calculating Break-Even Effects
Suppose three equally sized patient groups have estimated treatment effects of 0.08, 0.04, and 0.015 quality-adjusted life-years (QALYs) per person. For this hypothetical calculation, value one QALY at \(\lambda=\$40{,}000\), and suppose the corresponding incremental treatment costs are \(\$1{,}200\), \(\$1{,}000\), and \(\$600\) per person. Assume there is no additional burden, so \(h(x)=0\). The rule is to treat when \(\tau(x)>c(x)/\lambda\).
For group 1, the break-even effect is \(\$1{,}200/\$40{,}000=0.030\) QALYs. Its effect of 0.08 exceeds that threshold. Its incremental utility is \((\$40{,}000)(0.08)-\$1{,}200=\$2{,}000\) per person.
For group 2, the threshold is \(\$1{,}000/\$40{,}000=0.025\) QALYs. Its effect of 0.04 exceeds the threshold, and its incremental utility is \((\$40{,}000)(0.04)-\$1{,}000=\$600\) per person.
For group 3, the threshold is \(\$600/\$40{,}000=0.015\) QALYs. Its effect is exactly at break-even, so treatment and no treatment have equal expected incremental utility under this model. If the policy treats only when incremental utility is strictly positive, it does not treat this group. The tie is not evidence that treatment has no causal effect: the estimated effect is positive, but its valued benefit equals its cost.
Stability Across Costs and Outcome Valuations
A policy is stable over a set of assumptions if its treatment assignments do not change over that set. Its value can nevertheless change even when assignments remain fixed, because the monetary or utility value assigned to each outcome changes. Conversely, a small change in inputs can switch a group near its break-even point while leaving other groups unchanged.
For a group with a positive effect and no additional burden, the break-even valuation of the outcome is \(\lambda^*=c/\tau\). Below this valuation, the treatment's valued benefit is less than its cost; above it, the valued benefit exceeds cost. This is a useful way to report sensitivity: instead of presenting only one chosen value for \(\lambda\), report the value at which the decision changes. If effects or costs differ across groups, these switching values will differ too.
Worked Example: A Group-Level Stability Analysis
Consider four equally sized groups with hypothetical effects and treatment costs shown below. Effects are in QALYs per person, costs are dollars per person, and there is no additional burden. For each valuation, calculate \(\Delta_g=\lambda\tau_g-c_g\), treat when \(\Delta_g>0\), and calculate policy value as the average of the selected groups' incremental utilities across all four groups.
| Group | \(\tau_g\) (QALYs) | \(c_g\) (dollars) | \(\Delta_g\) at \(\lambda=\$25{,}000\) | \(\Delta_g\) at \(\lambda=\$30{,}000\) | \(\Delta_g\) at \(\lambda=\$40{,}000\) |
|---|---|---|---|---|---|
| A | 0.060 | 1,800 | -300 | 0 | 600 |
| B | 0.040 | 900 | 100 | 300 | 700 |
| C | 0.025 | 400 | 225 | 350 | 600 |
| D | 0.015 | 100 | 275 | 350 | 500 |
At \(\lambda=\$25{,}000\) per QALY, group A has net utility \((\$25{,}000)(0.060)-\$1{,}800=-\$300\), so it is not selected. Groups B, C, and D have positive net utilities of \(\$100\), \(\$225\), and \(\$275\), respectively. The policy value is \((\$100+\$225+\$275)/4=\$150\) per person in the full target population.
At \(\lambda=\$30{,}000\), group A is exactly at break-even: \((\$30{,}000)(0.060)-\$1{,}800=\$0\). Under the strict-positive rule it remains unselected. Groups B, C, and D are selected, giving value \((\$300+\$350+\$350)/4=\$250\) per person.
At \(\lambda=\$40{,}000\), group A has net utility \(\$600\), so all four groups are selected. The policy value is \((\$600+\$700+\$600+\$500)/4=\$600\) per person. Group A's decision switches at \(\lambda^*=\$1{,}800/0.060=\$30{,}000\) per QALY. For comparison, group B switches at \(\$900/0.040=\$22{,}500\), group C at \(\$400/0.025=\$16{,}000\), and group D at \(\$100/0.015\approx\$6{,}667\) per QALY. The policy is therefore more sensitive to valuation for group A than for the other three groups over this range.
Cost changes can be assessed in the same way. At \(\lambda=\$30{,}000\), suppose each group's treatment cost increases by \(\$200\). The new net utilities are \(-\$200,\ \$100,\ \$150,\ \$150\) for groups A through D. Groups B, C, and D remain selected, and policy value falls to \((\$100+\$150+\$150)/4=\$100\) per person. The selected set is unchanged from the original \(\$30{,}000\) scenario, but its value is lower.
Decision Curves and Thresholds
A decision curve summarizes policy value as a decision threshold varies. For a threshold \(r\), define the rule \(d_r(x)=1\) when \(\Delta(x)>r\), and 0 otherwise. Its value, measured against treating nobody, is
where \(\mathbb{1}\{\cdot\}\) equals 1 when the condition is true and 0 otherwise. A curve or table of \(V(r)\) shows how the selected population and the expected incremental value change with the minimum net gain required for treatment. The threshold \(r\) is a decision hurdle; unless explicitly included in the utility model, it is not an additional treatment cost and should not be subtracted from policy value.
Worked Example: Reading a Decision Curve
Suppose a hypothetical population consists of four equally sized groups whose incremental utilities, after accounting for treatment effects and costs, are \(-\$100\), \(\$100\), \(\$300\), and \(\$500\) per person. Use the rule \(\Delta>r\), and calculate policy value as the average incremental utility among all four groups, counting unselected groups as zero.
At \(r=-\$150\), all four groups are selected. Policy value is \((-\$100+\$100+\$300+\$500)/4=\$200\) per person. The negative-utility group reduces the value of this policy.
At \(r=\$0\), the group with \(-\$100\) is excluded and the other three are selected. Policy value is \((\$100+\$300+\$500)/4=\$225\) per person.
At \(r=\$200\), only the groups with \(\$300\) and \(\$500\) are selected, so value is \((\$300+\$500)/4=\$200\) per person. At \(r=\$400\), only the \(\$500\) group is selected, giving \(\$500/4=\$125\) per person. As the positive hurdle rises, fewer positive-contribution groups are treated and value falls in this example. The decision curve makes that loss visible rather than showing only the changing treatment rate.
A zero hurdle has a special interpretation under this model: it selects everyone with positive incremental utility and excludes everyone with negative incremental utility. Raising the hurdle above zero excludes some people who would have added positive value, so value cannot increase through that exclusion alone. Lowering the hurdle below zero can include people with negative incremental utility and reduce value. The curve can therefore rise as a negative threshold approaches zero and fall as a positive threshold increases. It is not generally a monotone curve over all possible thresholds.
Assumptions and Practical Interpretation
These calculations are only as useful as their inputs and decision model. The treatment effects must represent the target population and be on the stated outcome scale. The valuation \(\lambda\), costs, and burdens must be expressed consistently, and the additive utility equation must be a reasonable representation of the decision-maker's preferences. If costs vary within a group, use the relevant individual-level cost or a suitable conditional average rather than assuming all members face the same cost.
Sensitivity analysis is not a substitute for uncertainty analysis. The stability calculations above hold the listed effects fixed and vary valuation or cost; they do not account for sampling error in the effect estimates. A group that remains selected across a valuation range could still have an uncertain effect. Conversely, a decision that switches near one threshold is sensitive to the stated assumptions, but that alone does not show that the estimated causal effect is imprecise. Keep uncertainty in estimates distinct from sensitivity to choices about valuation and costs.
A hurdle also needs a clear interpretation. If \(r\) is an implementation requirement, show it as a decision threshold and report policy value using the underlying utility \(\Delta\). If the additional amount is genuinely a per-person expense, include it in treatment cost \(c\) instead. Treating a hurdle as a cost in one calculation and merely as a rule in another changes the meaning of the reported policy value.
Common Mistakes
- Using the effect alone as the treatment rule. A positive effect does not guarantee positive incremental utility. Compare the valued benefit \(\lambda\tau(x)\) with cost and burden.
- Changing the selected group without recalculating value. After a cost or valuation change, recalculate \(\Delta(x)\), apply the rule, and average contributions over the whole target population.
- Calling a decision hurdle a cost. A threshold \(r\) changes which people are selected. It reduces the utility of each treated person only if it represents an actual cost that belongs in the utility model.
- Interpreting a stable decision as a certain effect. Stability over selected input values describes robustness to those choices; it does not establish statistical certainty or causal identification.
Check Your Understanding
Use the definitions and decision rules above to answer these questions.
- A treatment effect is 0.02 outcome units, the outcome is valued at \(\$10{,}000\) per unit, and treatment costs \(\$150\) per person. What is the incremental utility, and does the zero-hurdle rule select treatment?
- For a positive treatment effect, how does increasing the outcome valuation change the break-even effect threshold when costs are fixed?
- In the four-group stability example, why is group A not selected at \(\lambda=\$30{,}000\) under the strict-positive rule?
- What does a decision curve report that a list of selected groups alone does not?
- Explain why varying a treatment cost while holding an estimated treatment effect fixed is a sensitivity analysis, not an analysis of uncertainty in that effect.