Boundary Adequacy Under Perturbation and Counterfactuals

Why Stress-Test a Boundary?

A model boundary specifies which biological processes and external influences are represented in the model and which are left outside it. A boundary is not adequate in the abstract; it is adequate, or not, for a defined decision, endpoint, population, and time horizon. Earlier material on diagnosing bias from omitted feedback loops highlights one reason an exclusion can matter: the omitted process may change the model’s predicted response. A stress test asks a more decision-focused question: could a plausible change in an included input, or a plausible effect of an excluded process, change the treatment ranking or the decision itself?

The test is a set of conditional model calculations, not a statistical test. It does not establish that a perturbed input value or excluded process occurs in patients. Its value is that it makes the consequence of the boundary visible: if a modest, credible perturbation changes the preferred intervention, then the decision depends on that boundary choice and should be reported as conditional or investigated further.

Set Up a Decision-Focused Test

Let \(Y(d,u)\) denote the model-predicted outcome under intervention \(d\) and input conditions \(u\). The input vector \(u\) may include such quantities as baseline disease state, a forcing input, or a contextual condition that the model already represents. For two interventions, define the paired contrast

The same input condition \(u\) must be used for both counterfactuals. Otherwise, the contrast combines the intervention difference with a change in context, making the treatment comparison difficult to interpret. The sign of \(\Delta\) identifies which intervention is preferred only after the outcome scale is defined. For example, if \(Y\) is benefit, a larger value may be better; if \(Y\) is disease burden, a smaller value may be better.

A useful stress test has four elements:

• Decision: state what choice the model is meant to inform, such as selecting between two interventions or deciding whether a predicted benefit clears a minimum criterion.
• Domain: define the population, context, endpoint, time horizon, and range of input conditions relevant to that choice.
• Perturbations: vary one or more decision-relevant inputs over a range supported by evidence or by the intended use domain. Label scenario values as hypothetical when they are illustrative.
• Interpretation: record whether the ranking or threshold decision changes, and identify which boundary assumption is responsible for the change.

For an excluded process, a counterfactual scenario can represent a bounded effect of that process on the output or treatment contrast. This is a stress-test device, not evidence that the omitted process has been correctly specified. In particular, the effect should not be selected merely because it produces a desired conclusion. Its range needs a scientific rationale, such as relevant evidence, a defensible upper bound, or an explicitly labelled exploratory scenario.

When Can a Ranking Reverse?

Suppose the baseline contrast is \(\Delta(u_0)\), and the input is moved along a specified perturbation direction \(v\), so that \(u(t)=u_0+t v\). If the response is locally approximated by a straight line, the first-order change in the contrast is the directional derivative \(s\). The approximation is then \(\Delta(u(t))\approx\Delta(u_0)+st\). If \(s\) has the opposite sign to the baseline contrast, the approximate crossing occurs at:

This follows by setting the approximated contrast equal to zero and solving \(\Delta(u_0)+st=0\). A crossing matters for boundary adequacy only if it falls within the justified perturbation range. The formula is a local or linear-scenario calculation; when the model response is nonlinear, evaluate the model at multiple points across the range rather than relying on the line approximation.

A threshold decision has the same structure. If a predicted benefit \(B(u)\) must meet a minimum value \(T\), define the margin \(M(u)=B(u)-T\). The decision changes when \(M(u)\) crosses zero. A baseline prediction just above \(T\) has little margin, so even a small plausible perturbation may change the decision. That is a reason to examine the boundary, not a reason to treat the threshold as a biological discontinuity.

Representing an Excluded Process as a Counterfactual

A process outside the model boundary cannot be directly varied as one of the model’s represented mechanisms. One practical stress test is to define a transparent scenario adjustment to the outcome or contrast and ask how large that adjustment would need to be to change the decision. The adjustment must have clear units and a specified direction. It is a probe of decision dependence, not a replacement for a mechanistic representation.

Thresholds, Diagnostics, and Failure Modes

Several checks help determine whether a stress test is informative rather than misleading:

• Check paired conditions. Compare interventions at the same population, input values, endpoint, and time horizon. A change in any of these can confound the comparison.
• Check directions and units. Confirm that “larger is better” or “smaller is better” is stated correctly, and that a scenario adjustment is expressed on the same scale as the outcome.
• Check the range. A crossing outside the intended domain does not demonstrate a decision reversal within that domain. A crossing inside an unjustified range is not persuasive evidence of a real vulnerability.
• Check the shape of the response. Sparse endpoints can miss curvature or a non-monotone response. Evaluate intermediate values when there is reason to expect nonlinearity.
• Check the boundary claim. A changed prediction shows dependence on the tested scenario. It does not, by itself, identify the true omitted biology or prove that adding more detail would improve the decision.

A common mistake is to perturb an included input, see no ranking change, and conclude that the boundary is adequate. That test only addresses the chosen input perturbation. It says nothing about a different excluded process that could affect the same contrast. The correction is to state separately which represented inputs were varied and which excluded-process counterfactuals were considered. Another mistake is to call a treatment “robustly preferred” because it wins at the baseline input. The correction is to report the range tested and whether the ranking holds across that range.

These tests complement, but do not replace, other analyses. A local sensitivity calculation describes how an output changes near a reference condition; a finite-range stress test checks whether the decision changes over a specified domain. Parameter uncertainty analysis asks how uncertainty in estimated quantities affects predictions, whereas an excluded-process counterfactual probes a structural omission. Comparing a small number of justified scenarios is not a comprehensive assessment of every plausible mechanism or model structure. Finally, a stress test can reveal a fragile decision, but it cannot show that a boundary change is feasible, supported by evidence, or beneficial.