Why Stress-Test a Decision?
A decision can look satisfactory under a reference model and still depend on assumptions that are uncertain or unfavorable. A stress test asks a focused question: does the proposed action continue to meet its decision criteria under adverse combinations of assumptions that remain defensible? In model-informed drug development, the action might be advancing a candidate dose, continuing development, or proceeding with a particular trial design. The relevant outcomes could include efficacy, exposure, or a safety measure.
Stress testing is related to sensitivity analysis, but it has a decision-focused purpose. Rather than asking only how much an output changes when an input changes, it evaluates whether that change crosses a prespecified boundary for action. A stress test is not automatically a forecast of what is likely to happen, and a deliberately adverse scenario should not be described as the most probable scenario.
Specify the Decision and the Stress Set
Write the action as \(a\), the uncertain model inputs as \(\theta\), and the decision-relevant model output as \(g(a,\theta)\). A criterion might require an efficacy measure to exceed a minimum, a safety measure to remain below a maximum, or both. For example, if \(E(a,\theta)\) is a predicted efficacy outcome and \(S(a,\theta)\) a predicted safety outcome, the action meets the criteria in scenario \(\theta\) when \(E(a,\theta)\geq E_{\min}\) and \(S(a,\theta)\leq S_{\max}\).
The stress set \( \mathcal{S} \) should be specified before evaluating the action. It can be defined by defensible parameter ranges, selected joint scenarios, or both. Its justification should come from the scientific context: for example, uncertainty in parameter estimates, credible biological bounds, sensitivity to model assumptions, or plausible variation in a target population. A range chosen only because it produces a desired conclusion is not a defensible stress set.
Several assumptions are needed for the exercise to be informative. The model must be suitable for the action and outcome being evaluated, and it must be used within a domain where its predictions are interpretable. The stress scenarios must be scientifically defensible, including their joint combinations. Finally, the decision criteria and relevant time horizon must be explicit. A stress test cannot repair a misspecified model, justify unsupported input ranges, or determine the decision criteria on its own.
The stress set is not necessarily a probability distribution. If scenarios are evaluated without probability weights, their results establish whether the decision passes across the chosen set; they do not establish the probability of failure. Probability-of-target analyses and predictive uncertainty summaries answer different questions and should not be inferred from a deterministic stress test.
Specify what is being considered and the outcome thresholds that define an acceptable result.
Give the input ranges or joint scenarios, explain why they are plausible, and identify any parameter dependence or constraints.
Use the model to calculate outcomes at the relevant scenarios and compare each outcome with the criteria.
Describe which assumptions, combinations, or thresholds lead to an unacceptable outcome, and state the limitations of the analysis.
Worked Examples
Worked Example: Testing Efficacy and Exposure at a Candidate Dose
Suppose a hypothetical candidate dose of 40 mg is being evaluated. For illustration, predicted exposure is \(C=DF/CL\), where \(D\) is dose in mg, \(F\) is a dimensionless availability factor, and \(CL\) is clearance in L/h. The model’s exposure units are treated as the relevant units for this hypothetical decision. The assumed input ranges are \(0.70\leq F\leq0.90\) and \(3.5\leq CL\leq5.0\) L/h, and the analysis assumes the combinations formed by these ranges are defensible. Predicted efficacy is \(R=80C/(5+C)\) response units. The criteria are \(R\geq45\) response units and \(C\leq10\) exposure units.
Reference case. At \(F=0.80\) and \(CL=4.0\) L/h, predicted exposure is
The predicted efficacy is \(R=80(8)/(5+8)=640/13=49.23\) response units. Both criteria are met in this reference case.
Adverse case for efficacy. Lower availability and higher clearance reduce exposure. At \(F=0.70\) and \(CL=5.0\) L/h,
The efficacy criterion is not met because \(42.26<45\) response units. This combination is adverse for efficacy within the stated ranges.
Adverse case for the exposure limit. Higher availability and lower clearance increase exposure. At \(F=0.90\) and \(CL=3.5\) L/h,
This exceeds the limit of 10 exposure units. The dose therefore fails the stress test: the reference case passes, but the efficacy and exposure criteria each fail under an adverse scenario. These results do not say how likely either scenario is. They identify the consequences of the stated assumptions if those scenarios are considered defensible.
Worked Example: Respecting Joint Plausibility
A second hypothetical analysis uses the same exposure and efficacy equations, dose, and criteria, but the relationship between availability and clearance is constrained by the model assumptions. Suppose the defensible joint scenarios are \((F,CL)=(0.70,3.9)\), \((0.80,4.1)\), and \((0.90,4.3)\), with clearance in L/h. Evaluating every combination of the marginal ranges instead would add combinations that this analysis regards as implausible.
At the three specified scenarios, exposure is:
The efficacy model \(R=80C/(5+C)\) increases with \(C\), since its derivative with respect to \(C\) is \(400/(5+C)^2>0\). Thus the lowest efficacy across these scenarios occurs at \(C_1=7.179\):
The highest exposure among the three scenarios is \(8.372\). The minimum efficacy exceeds 45 response units, and the maximum exposure is below 10 exposure units. The action therefore meets both criteria across this specified joint stress set.
If one instead combined \(F=0.70\) with \(CL=4.3\) L/h, exposure would be \(40(0.70)/4.3=6.512\), giving efficacy \(80(6.512)/(5+6.512)=45.25\) response units. That combination would fail the efficacy criterion of 46 response units if that stricter threshold were used, even though it is excluded by the specified relationship between the inputs. The lesson is not to discard adverse combinations automatically; it is to justify whether they belong in the stress set. Marginal ranges alone do not establish which joint combinations are plausible.
Deriving a Decision Boundary
A stress test can do more than list scenario outcomes. When a model is simple enough, algebra can show which range of actions could satisfy the criteria throughout the stress set. This boundary calculation is useful because it explains the failure mechanism rather than merely identifying a failed scenario.
For the exposure-response model in the first example, suppose \(R=80C/(5+C)\), with positive exposure \(C\). The efficacy threshold is 45 response units. Solving the inequality gives:
The equivalence uses \(5+C>0\), which holds for positive exposure. The safety condition is \(C\leq10\). Therefore, the acceptable exposure interval is \(6.429\leq C\leq10\).
Worked Example: Checking Whether Any Dose Passes Both Stress Criteria
Return to the first example’s independent ranges, \(0.70\leq F\leq0.90\) and \(3.5\leq CL\leq5.0\) L/h. For a positive dose, exposure \(C=DF/CL\) is smallest at \(F=0.70\), \(CL=5.0\), and largest at \(F=0.90\), \(CL=3.5\). To meet the efficacy criterion throughout the stress set, the smallest exposure must be at least \(45/7\):
To meet the exposure limit throughout the same set, the largest exposure must be no more than 10:
There is no dose satisfying both bounds: a dose would need to be at least 45.92 mg for the worst-case efficacy criterion and at most 38.89 mg for the exposure limit. Under this model, these criteria, and this stress set, no dose passes the full stress test. That conclusion does not establish that the candidate has no development path. It identifies a conflict that may motivate investigating other assumptions, measures, dose strategies, or decision criteria, with scientific justification for any changes.
Diagnostics, Alternatives, and Limitations
A stress test is only as credible as its model, stress set, and decision rule. Check the calculations at boundary scenarios, confirm that adverse directions are understood, and verify that every combination being evaluated is scientifically defensible. For a nonlinear model, do not assume that the most adverse outcome occurs at a corner of the input ranges; inspect the model over the stress set or justify mathematically why the boundary scenarios are sufficient. If a parameter’s effect changes direction across its range, evaluating only its endpoints can miss an interior failure.
Look for signs that the analysis is misleading: a stress set with no rationale, inconsistent units, an outcome threshold introduced after seeing the results, or a “worst case” assembled from marginal extremes that cannot jointly occur. Also check whether the model is being extrapolated beyond the data or intended domain. If the decision changes sharply under a small defensible change in an assumption, report that fragility rather than presenting only the favorable scenario.
| Approach | Main question answered | Important distinction |
|---|---|---|
| Stress testing | Does the action meet criteria across specified adverse scenarios? | Scenario coverage does not by itself assign probabilities. |
| Local sensitivity analysis | How does an output change near a reference case as an input changes? | A local change need not reveal threshold failure under large changes. |
| Probabilistic uncertainty analysis | How are predicted outcomes distributed under specified uncertainty distributions? | Results depend on probability models, not just a stress set. |
| Model comparison | How does the decision vary across candidate model structures? | Alternative models and adverse parameter scenarios are distinct sources of uncertainty. |
These approaches can complement one another. A stress test makes decision vulnerabilities visible, while probabilistic analyses can characterize uncertainty under explicit probability assumptions and model comparison can examine structural alternatives. Neither a pass nor a failure under a chosen stress set proves that the assumptions cover every relevant future condition. A pass means only that the stated criteria hold over the tested set, conditional on the model and its assumptions.
Common Pitfall
Wrong reasoning: “The decision passes in the central case, so it is robust.” A central case evaluates one combination of assumptions; it says nothing about other plausible combinations. Another common error is to call the most adverse constructed scenario “likely” merely because it is adverse.
Correction: Define the stress set and criteria explicitly, evaluate the action across the set, and report which scenarios pass or fail. Use probability language only when probability weights or a probability model support it. State the limitations of the model and the scenario set alongside the conclusion.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a stress test, an action meets the criterion in the reference case but fails in one included adverse scenario. Is it robust to the specified stress set?
- For \(C=DF/CL\), what happens to exposure as \(F\) increases, holding \(D\) and \(CL\) fixed? What happens as \(CL\) increases, holding \(D\) and \(F\) fixed?
- Why might combining the minimum of one parameter’s marginal range with the maximum of another parameter’s marginal range be misleading?
- With \(R=80C/(5+C)\), what is the minimum exposure required for \(R\geq45\) response units?
Solutions. (1) No. Robustness to a specified stress set requires meeting the criteria in every included scenario, so the action fails that test. (2) Exposure increases with \(F\) and decreases with \(CL\), since \(C\) is directly proportional to \(F\) and inversely proportional to \(CL\), for positive inputs. (3) The two extremes may be dependent or otherwise jointly implausible; evaluating them together can create a scenario outside the defensible stress set. (4) Solving \(80C/(5+C)\geq45\) gives \(80C\geq225+45C\), hence \(C\geq225/35=45/7=6.429\) exposure units.