Why compare development scenarios?
A development decision may depend on assumptions that are not yet settled: how often patients will take treatment as prescribed, what background therapy they will receive, or how long a treatment effect will persist. A single prediction can conceal how much the decision relies on these assumptions. Scenario analysis makes the alternatives explicit and shows what the model predicts under each one.
A scenario is a specified set of assumptions, not a forecast with an implied probability. For example, a “lower adherence” scenario answers, “What would the outcome be if adherence were lower, with the other assumptions in this scenario held as stated?” It does not answer how likely lower adherence is. The earlier material on propagating parameter uncertainty addresses uncertainty represented through distributions; scenario analysis instead compares selected, coherent alternatives.
For this tutorial, consider a simplified decision model for one patient over a fixed follow-up horizon. Let \(r_B\) be the event risk under the specified background therapy without the new treatment, \(\theta\) the relative reduction in event risk during effective treatment exposure, \(a\) the adherence fraction while a patient remains on treatment, and \(d\) the fraction of the planned follow-up during which the patient persists with treatment. Let \(C_F\) be the treatment cost for full planned exposure, and let \(\lambda\) be the monetary value assigned to one event avoided. These are hypothetical quantities for illustration, not estimates from a real study.
Under the illustrative model, adherence and persistence jointly determine the fraction of planned treatment exposure. The event risk with treatment is \(r_B(1-ad\theta)\). The expected number of events avoided per patient, relative to the same background therapy without the new treatment, is \(r_Bad\theta\). If cost is proportional to exposure, incremental cost is \(C_Fad\), and net monetary benefit (NMB) is the monetary value of events avoided minus incremental cost:
Here \(r_B\), \(a\), \(d\), and \(\theta\) are dimensionless proportions, while \(\lambda\) and \(C_F\) are in currency per patient-event and currency per full-exposure patient, respectively. Thus NMB is in currency per patient. A positive NMB favors the treatment under this simplified decision rule; it does not by itself establish clinical effectiveness, safety, or overall development value.
Assumptions that make the comparison interpretable
Before calculating scenario outcomes, specify what the model means by each input and what is held fixed. In the example model, the following assumptions are required:
- The event risk \(r_B\) applies to the stated population, follow-up horizon, and background therapy in that scenario.
- The relative event-risk reduction \(\theta\) applies during effective treatment exposure and is constant over the modeled period.
- Adherence \(a\) is the fraction of prescribed doses taken while treatment is ongoing; persistence \(d\) is the fraction of planned follow-up on treatment. Their product represents effective exposure under the model’s simplifying assumptions.
- Partial exposure scales the treatment effect linearly, and treatment cost scales linearly with exposure. If either relationship is not credible, the model must use a more appropriate exposure-effect or cost formulation.
- Outcomes are compared against the same background therapy within each scenario. Other outcomes, adverse events, discounting, and costs are omitted here and would need inclusion if decision-relevant.
The alternatives should also be internally coherent. A background-therapy assumption may change the untreated event risk, and it may also change the new treatment’s relative effect if there is a mechanistic or empirical reason to expect effect modification. Keeping \(\theta\) fixed is a modeling choice in the examples below, not a universal rule. If evidence supports a different treatment effect under a background regimen, the scenario should state and implement that change rather than silently importing the reference-case effect.
Worked Example: Deriving the Outcome for a Reference Case
Suppose the hypothetical reference case has \(r_B=0.20\), \(\theta=0.30\), \(a=0.80\), and \(d=0.75\). Full-exposure treatment cost is \(C_F=\$900\) per patient, and the decision value is \(\lambda=\$20{,}000\) per event avoided. Assume the model conditions above hold.
First calculate effective exposure: \(ad=0.80\times0.75=0.60\). The treated event risk is \(0.20[1-(0.60)(0.30)]=0.164\), or 16.4%. The background-therapy event risk is 20.0%, so the absolute reduction is \(0.200-0.164=0.036\), or 3.6 percentage points.
The expected events avoided per patient are \(r_Bad\theta=(0.20)(0.60)(0.30)=0.036\). Incremental cost is \(C_Fad=(\$900)(0.60)=\$540\) per patient. Therefore:
Under the stated rule, the reference-case NMB is positive by \$180 per patient. This is a conditional model result under the listed assumptions, not a statement that the treatment has a particular probability of being cost-effective.
Build and compare explicit alternatives
A useful scenario set begins with a reference case, then defines a small number of alternatives tied to meaningful development or clinical assumptions. In the table below, the background-therapy assumption changes the event risk without the new treatment. The treatment effect \(\theta\) is held at 0.30 in every scenario so the illustration isolates the stated differences in background risk, adherence, and persistence. These are joint scenario bundles; the values are not proposed as plausible bounds or assigned probabilities.
Worked Example: Calculating a Three-Scenario Comparison
Use the cost and value assumptions from the reference-case example: \(C_F=\$900\) and \(\lambda=\$20{,}000\). Consider the following explicitly hypothetical scenarios.
| Scenario | Background event risk \(r_B\) | Adherence \(a\) | Persistence \(d\) | Effect \(\theta\) |
|---|---|---|---|---|
| Reference case | 0.20 | 0.80 | 0.75 | 0.30 |
| Lower exposure, lower background risk | 0.12 | 0.60 | 0.50 | 0.30 |
| Higher exposure, higher background risk | 0.25 | 0.90 | 0.90 | 0.30 |
For the lower-exposure scenario, \(ad=(0.60)(0.50)=0.30\). Events avoided are \((0.12)(0.30)(0.30)=0.0108\) per patient, and incremental cost is \((\$900)(0.30)=\$270\) per patient. Its NMB is \((\$20{,}000)(0.0108)-\$270=\$216-\$270=-\$54\) per patient. The treated event risk is \(0.12[1-(0.30)(0.30)]=0.1092\), or 10.92%; the absolute reduction is 1.08 percentage points.
For the higher-exposure scenario, \(ad=(0.90)(0.90)=0.81\). Events avoided are \((0.25)(0.81)(0.30)=0.06075\) per patient, and incremental cost is \((\$900)(0.81)=\$729\) per patient. Its NMB is \((\$20{,}000)(0.06075)-\$729=\$1{,}215-\$729=\$486\) per patient. The treated event risk is \(0.25[1-(0.81)(0.30)]=0.18925\), or 18.925%; the absolute reduction is 6.075 percentage points.
Across these three specified scenarios, the reference and higher-exposure cases have positive NMB, whereas the lower-exposure case has negative NMB. The event risk with treatment is higher in the higher-background-risk scenario than in the reference case, even though its absolute reduction and NMB are also higher. This is not a contradiction: the model’s absolute benefit depends on both baseline risk and effective exposure. No scenario is thereby shown to be more likely than another.
Check that the assumptions and arithmetic travel together
Scenario analysis is only informative when the input changes and the outcome calculation refer to the same definition and population. For example, the background event risk must correspond to the background regimen named for that scenario. If a new regimen affects adherence, persistence, or treatment effect as well, those changes should be stated, not implicitly attributed to the background event risk alone.
A particularly important consistency check concerns adherence and persistence. Multiplying them is appropriate in the simple model only when adherence describes dose-taking during time on treatment and persistence describes the fraction of planned time on treatment. If a data source instead reports “adherence” as the proportion of all planned doses taken, including time after discontinuation, multiplying that measure by persistence can count discontinuation twice. Definitions must be checked before inputs are combined.
The model also assumes costs fall in direct proportion to exposure. That may be reasonable for some per-dose costs, but not for a fixed treatment-course price, initiation costs, monitoring, or costs that continue after discontinuation. The decision metric should use the cost structure appropriate to the decision, not whichever formula is easiest to calculate.
Worked Example: A Fixed-Cost Assumption Changes the Decision Result
Return to the reference case, where effective exposure is \(ad=0.60\) and events avoided are 0.036 per patient. Suppose, contrary to the earlier proportional-cost assumption, the full treatment-course cost of \$900 is incurred regardless of adherence or persistence. The benefit calculation remains \((\$20{,}000)(0.036)=\$720\) per patient, but cost is now \$900 per patient rather than \((\$900)(0.60)=\$540\).
The NMB under this fixed-cost assumption is \(\$720-\$900=-\$180\) per patient. Under proportional cost it was \(+\$180\) per patient. The sign changes because the two cost assumptions are different, not because the clinical event calculation changed. A scenario comparison must therefore identify which costs vary with adherence and persistence and which are fixed; presenting one cost rule as if it were a clinical fact would misrepresent the decision.
Justification and limits of the simplified calculation
The event calculation follows directly from the model’s exposure assumption. During the fraction \(ad\) of effective exposure, risk is reduced by the relative fraction \(\theta\); during the remaining fraction \(1-ad\), risk is \(r_B\). Averaging these risks gives:
Subtracting this risk from the background risk \(r_B\) gives \(r_B-r_B(1-ad\theta)=r_Bad\theta\), the events avoided per patient in the model. This derivation relies on the simplifying assumption that exposure fractions can be averaged in this way and that risk reduction scales linearly with exposure. If event risk varies over time, treatment effect has delayed onset or waning, or persistence depends on prognosis, a time-to-event or other suitable model may be needed instead.
Scenario analysis is useful because it exposes decision dependence on specified assumptions in an auditable form. It is not a substitute for estimating uncertain parameters, evaluating model discrepancy, or assigning probabilities to future outcomes. Those are different tasks. Nor does a small scenario set cover every possible combination: it is a deliberately selected set of conditional calculations.
Compared with a single reference prediction, scenario analysis makes assumptions and consequences more transparent. Compared with probabilistic analysis, it does not average across a distribution of inputs or quantify the probability of a decision outcome. Compared with changing one input at a time, a joint scenario can represent a coherent package of assumptions, but it may not reveal which input drove the difference. Avoid attributing the entire outcome change to one assumption when several changed together.
Useful diagnostics include checking that all scenarios use the same population, horizon, outcome definition, and decision rule; confirming that probabilities and fractions fall in their valid ranges; reproducing the reference case from its stated inputs; and verifying that displayed outcomes follow the equations. Large or counterintuitive changes warrant checking the input mapping, units, and model structure before interpreting the results. A scenario that produces an impossible event risk, for example a value outside 0 to 1, signals a model or input problem rather than a novel clinical finding.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In the illustrative model, \(r_B=0.10\), \(a=0.50\), \(d=0.80\), and \(\theta=0.25\). Calculate events avoided per patient.
- Why is it incorrect to interpret three scenario outcomes as probabilities that the decision will have those outcomes?
- If adherence is already defined as the fraction of all planned doses taken, including after discontinuation, what needs checking before multiplying it by persistence?
- A scenario changes background risk, adherence, and persistence together. Can the outcome difference be attributed to adherence alone?
Solutions. (1) Events avoided are \(r_Bad\theta=(0.10)(0.50)(0.80)(0.25)=0.010\) per patient, or 1.0 event per 100 patients on average under the model. (2) Scenarios are conditional calculations; without justified probabilities assigned to them, they do not quantify how likely each outcome is. (3) Check whether discontinuation is already reflected in the adherence measure; if so, multiplying by persistence may count time off treatment twice. (4) No. Because multiple assumptions changed, the result is a comparison of the full scenario bundles, not an isolated estimate of the adherence contribution.