Why Stress-Test an Incomplete Chain?
An integrated evidence chain may connect mechanistic assumptions, pharmacokinetic predictions, exposure-response relationships, and trial outcomes. A component can be absent, represented by a weakly supported estimate, or applicable only under conditions different from those of interest. A single preferred input can conceal how much the final prediction depends on that component. Stress-testing makes this dependence visible: it asks whether the decision-relevant output remains acceptable across defensible alternatives.
The aim is not to make an incomplete chain appear complete. It is to identify which conclusions follow from the supported evidence, which depend on assumptions, and what evidence would most reduce the uncertainty. A stress test is informative only when its alternatives have a scientific rationale and are propagated through the same chain as the primary analysis.
A stress test requires a clear output, such as exposure, probability of response, or an expected trial-level outcome. It also requires a defined component to vary and a reason for the range being considered. Before using the method, check that the chain's equations and units are coherent; that fixed inputs are genuinely held fixed; that the alternatives are plausible in the target setting; and that dependencies between uncertain components have not been discarded. A range is not automatically a probability distribution, and its endpoints are not automatically equally likely.
From a Weak Component to an Output Range
Let \(q\) denote a missing or weakly supported component, \(x\) the other inputs, and \(Y=f(q,x)\) an integrated output. A bounded stress test specifies \(q\in[q_L,q_U]\), holds \(x\) at stated values or varies it in a prespecified way, and evaluates \(f\) under the resulting scenarios. The range of outputs describes sensitivity to those assumptions; by itself it does not describe the probability that the true output lies within the range.
A useful justification applies when the output is monotone in the uncertain component. If \(f(q,x)\) is increasing in \(q\) throughout the interval, then for every admissible \(q\), \(q_L\le q\le q_U\) implies \(f(q_L,x)\le f(q,x)\le f(q_U,x)\). Thus the endpoint outputs bound all outputs in that interval. If the relationship is decreasing, the endpoint order reverses. This argument does not apply if the function is nonmonotone, if assumptions interact, or if changing one input also changes another; in those cases, evaluate interior values or a suitably designed joint scenario set.
The bounds should come from relevant evidence, scientific constraints, or explicit expert assumptions. If there is no defensible finite range, report that limitation instead of selecting convenient endpoints. When multiple components are uncertain, preserve known relationships between them. Combining the low value of one component with the high value of another may create an impossible scenario if those inputs are physiologically or statistically dependent.
Worked Example: Bounding Exposure When Bioavailability Is Weakly Supported
Suppose a hypothetical oral drug is given at 100 mg per dosing interval. Under a linear, time-invariant pharmacokinetic model, the interval AUC is \(AUC=F D/CL\), where \(F\) is bioavailability, \(D\) is dose, and \(CL\) is clearance. Assume clearance is fixed at 10 L/h. Bioavailability has limited support in the target population, so the team prespecifies \(0.40\le F\le0.80\). These are scenario bounds, not a probability distribution.
The model assumes the same dose and clearance across scenarios, and that the stated equation is appropriate for the interval AUC. For fixed positive \(D\) and \(CL\), AUC increases with \(F\), so the endpoint calculation bounds the results:
At a reference value \(F=0.60\), the predicted AUC is \(0.60(100)/10=6.00\ \mathrm{mg\cdot h/L}\). If a hypothetical target is at least \(6.00\ \mathrm{mg\cdot h/L}\), the reference scenario meets it, but the lower-bound scenario does not. The chain therefore does not support a robust claim that the target is met across the prespecified range. This calculation does not establish which bioavailability value is true; it shows that better support for \(F\), or a decision that tolerates the lower prediction, matters to the conclusion.
Propagate Alternatives Through the Chain
The component under stress may sit upstream of several transformations. It is not enough to describe its local effect if the decision depends on a downstream quantity. Propagate each alternative through the remaining models and report the output in units that can be interpreted for the decision. Hold other inputs fixed only when that is scientifically meaningful, and state which ones are fixed.
If the output is nonmonotone, calculate more than the endpoints. A simple grid can reveal an interior maximum or minimum; a joint scenario set can represent combinations that are plausible together. For probabilistic analyses, an uncertain component may instead be represented by a distribution and propagated with its stated dependencies. That answers a different question from bounded scenarios: probability statements require justified probability distributions, not just ranges. A bounded analysis can be used alongside probabilistic propagation to probe tails or structural alternatives, but the two should not be conflated.
Worked Example: Testing a Weak Exposure-Response Slope
Consider a hypothetical exposure-response model for the probability of response, \(p\), with \( \operatorname{logit}(p)=-1+\beta(AUC-4)\), where \( \operatorname{logit}(p)=\log\{p/(1-p)\}\). At \(AUC=8\ \mathrm{mg\cdot h/L}\), the slope \(\beta\) is weakly supported. The prespecified plausible range is \(0.15\) to \(0.35\) per \(\mathrm{mg\cdot h/L}\). Assume the model form and intercept are held fixed and that this range is scientifically admissible.
For \(\beta=0.15\), the linear predictor is \(-1+0.15(8-4)=-0.40\). Therefore \(p=1/(1+e^{0.40})=0.4013\). For \(\beta=0.35\), it is \(-1+0.35(4)=0.40\), giving \(p=1/(1+e^{-0.40})=0.5987\). Because \(AUC-4\) is positive here and the logistic function increases with its linear predictor, the probability is increasing in \(\beta\); the endpoint values bound the probabilities over the stated slope range.
For a hypothetical trial with 100 evaluable participants, the expected number of responders is \(100p\), or 40.13 to 59.87. This is an expected count range under the alternative slopes, not a prediction interval for the realized count. The chain's expected response can therefore shift by about 20 participants across these assumptions. The calculation assumes a common response probability for the evaluable participants; it does not establish that the slope bounds are equally likely or that the model is adequate.
Use Predictive Checks to Look for Tension
A stress test asks how outputs change under assumptions; a predictive check asks whether a prediction is compatible with an observed quantity relevant to the chain. These are complementary. A predictive discrepancy can flag that a chain, its inputs, or its uncertainty representation may be inadequate, but it generally cannot identify which component caused the mismatch. For the check to be interpretable, the observed quantity should be relevant to the predicted output and should not have already been used as if it were independent validation evidence.
For a normally distributed predictive quantity with predictive mean \(m\) and predictive variance \(s_{\mathrm{pred}}^2\), suppose an independent observed estimate \(y\) has measurement standard error \(SE\). If predictive and measurement errors are independent, both are normal, and the variances are appropriately specified, the difference \(y-m\) has variance \(s_{\mathrm{pred}}^2+SE^2\). A standardized discrepancy is therefore \(z=(y-m)/\sqrt{s_{\mathrm{pred}}^2+SE^2}\). This calculation is a diagnostic under those assumptions, not a general-purpose proof that the model is correct or incorrect.
Worked Example: Checking an Integrated Prediction Against External Data
A hypothetical integrated chain predicts an external biomarker summary with mean \(m=42\) units and predictive standard deviation \(s_{\mathrm{pred}}=5\) units. An independent external estimate is \(y=54\) units with measurement \(SE=3\) units. Assume normal predictive and measurement errors, independence, and that the external summary is on the same scale and target population as the prediction.
The combined standard deviation for the difference is \(\sqrt{5^2+3^2}=\sqrt{34}=5.831\) units. Hence \(z=(54-42)/5.831=2.058\). Under the stated normal reference distribution, the two-sided tail area is approximately \(2\Phi(-2.058)=0.0396\). The observation is about 2.06 combined standard deviations above the predictive mean, a signal of tension worth investigating. The tail area is not the probability that the chain is wrong, nor does it identify a faulty component.
Suppose an alternative, scientifically plausible setting for one weak component shifts the predictive mean to 49 units while leaving the predictive standard deviation at 5 units. Its discrepancy is \((54-49)/\sqrt{34}=0.8575\), with two-sided normal tail area about 0.391. This alternative is more compatible with the observed summary under the stated reference calculation. It does not prove that this setting is correct: the shift might reflect an incorrect input, a missing mechanism, or a mismatch between the external data and the target setting. The assumptions and the data's independence from model development must be reviewed before treating this as a validation check.
Diagnostics, Alternatives, and Limitations
A stress test is most useful when it shows both the range of outputs and the decision consequence. Check whether conclusions change across the prespecified alternatives, whether outputs behave as expected as an input varies, and whether any output violates basic constraints or becomes implausible. An abrupt or counterintuitive shift can indicate a coding, unit, interface, or structural problem; a smooth response does not by itself establish model adequacy. Where the output depends on several uncertain components, inspect interactions rather than assuming one-at-a-time changes capture the joint uncertainty.
| Approach | Main question | Key limitation |
|---|---|---|
| Bounded scenarios | Does the conclusion hold across a specified plausible range? | Bounds do not assign probabilities to outcomes. |
| Probabilistic propagation | What output distribution follows from specified input distributions? | Results depend on justified distributions and dependencies. |
| Predictive check | Is an observed quantity compatible with the chain's predictive distribution? | A discrepancy does not identify its cause. |
These approaches answer different questions and can be used together. One-at-a-time sensitivity analysis is useful for locating influential inputs, but it can miss interactions and combinations that matter jointly. A bounded scenario analysis makes assumptions visible, but its output range depends on the chosen bounds. Probabilistic propagation summarizes uncertainty only to the extent that distributions, dependencies, and model structure are credible. Predictive checks can reveal mismatch, but an apparently compatible observation does not prove that every component is sound.
Avoid replacing a weak component with an arbitrary value merely because it produces a convenient result. Also avoid treating the full range as a confidence interval or claiming a probability of success from endpoints alone. If the decision changes over plausible alternatives, report that lack of robustness and identify the component or additional data most relevant to resolving it. If it does not change, the decision is robust to the tested assumptions, not necessarily to every untested structural uncertainty.
Name the quantity, its units, and the decision threshold or interpretation that matters.
Justify bounds or scenarios using relevant evidence, and state dependencies and fixed inputs.
Use endpoint calculations only when monotonicity justifies them; otherwise evaluate interior and joint scenarios as needed.
Report whether the decision changes, inspect implausible behavior, and use relevant independent observations to probe predictive compatibility.
Separate conclusions supported across scenarios from those dependent on weak assumptions, and do not assign probabilities without a justified probability model.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A model output increases with a weak input \(q\), which is bounded from 2 to 5. What calculation can bound the output if all other inputs are fixed?
- Why is an output range from bounded scenarios not automatically a 95% interval?
- A predictive check shows a large discrepancy. What can it establish, and what can it not establish on its own?
- When should endpoint calculations be supplemented by interior or joint scenarios?
Solutions. (1) Evaluate the output at \(q=2\) and \(q=5\); monotonic increase ensures these are the lower and upper outputs when the other inputs are fixed. (2) The bounds specify admissible assumptions, not a sampling procedure or probability distribution, so they do not confer 95% coverage. (3) It can flag tension between an observation and the predictive distribution under the stated assumptions; it cannot by itself show which component is responsible or prove the chain is wrong. (4) Do so when the output is nonmonotone, uncertain inputs interact, or dependencies make endpoint combinations insufficient or implausible.