Why Test Interface Assumptions?
An integrated prediction is produced by linking model components, and each link requires assumptions about how information passes between them. A parameter may be transferred from one model to another; an exposure quantity may be defined differently across components; a model population may not match the target population; or a dependence structure may be simplified. These choices can affect the final prediction even when the component models themselves are unchanged.
Testing sensitivity to interface assumptions means deliberately changing a defensible interface choice and evaluating how the integrated prediction responds. The purpose is not to find a statistically significant difference or to prove that one assumption is correct. It is to identify which choices matter to the decision, whether the direction and size of the change are understandable, and whether the conclusion is robust to plausible alternatives.
Before comparing scenarios, define the prediction of interest (for example, response probability at a dose, expected biomarker change, or probability of meeting a trial endpoint), the target population and time horizon, and the baseline interface choices. Alternatives should be plausible and interpretable, not arbitrary extreme values. Use the same model structure, data inputs, and computational procedure in each comparison except for the intended interface change.
A Controlled Comparison
For a baseline prediction \(g_0\) and a prediction \(g_j\) under an alternative assumption, the absolute change is \(\Delta_j=g_j-g_0\). When a relative change is meaningful and \(g_0\ne 0\), it can also be reported as \(100(g_j-g_0)/g_0\) percent. The scale matters: for a probability, an absolute change of 0.05 is a change of 5 percentage points, not necessarily a 5 percent relative change.
A one-at-a-time analysis changes one interface assumption while holding the others at baseline. It makes the effect of that particular choice easy to interpret, but it does not reveal interactions between assumptions. A joint scenario changes two or more assumptions together; it can be more realistic when assumptions are linked, but the resulting difference cannot generally be attributed to just one of them.
Worked Example: Transferring an Exposure-Response Parameter
Suppose a hypothetical integrated chain predicts response at a target dose using a logistic model. Its linear predictor is \(\eta=-2+0.84\theta\), where \(\theta\) is a dimensionless exposure-response parameter transferred from another model. The baseline transfer uses \(\theta=1\). A scientifically plausible alternative transfer uses \(\theta=2/3\). All other inputs are fixed. The predicted response probability is \(p=1/(1+\exp(-\eta))\).
At baseline, \(\eta=-2+0.84(1)=-1.16\), so \(p_0=1/(1+\exp(1.16))\approx 0.2387\), or 23.87%.
Under the alternative transfer, \(\eta=-2+0.84(2/3)=-1.44\), so \(p_1=1/(1+\exp(1.44))\approx 0.1915\), or 19.15%.
Calculate the difference from the unrounded model predictions: \(p_1-p_0\approx -0.04712\), which rounds to \(-0.0471\). Thus this alternative parameter transfer lowers the predicted response probability by about 4.71 percentage points. The result shows sensitivity to the transfer choice; it does not establish which value of \(\theta\) is correct.
Exposure Definitions and Population Alignment
An exposure-response component may use a particular exposure summary, such as concentration at a specified time, average concentration, or area under the concentration-time curve. A linked component must receive an exposure quantity with a compatible definition, units, time window, and population interpretation. If the definition changes, apply the same change consistently: do not substitute a new exposure summary while silently retaining a coefficient calibrated to the old scale.
Worked Example: Changing the Exposure Summary
In a hypothetical model, the probability of response is \(p=1/(1+\exp(-\eta))\), with \(\eta=-2+0.08E\). The exposure summary \(E\) is measured in a specified exposure unit. The baseline definition gives \(E=10\) units; an alternative, clinically defensible exposure definition gives \(E=7.5\) units. For this comparison, assume the coefficient 0.08 is defined for the same units and compatible exposure scale, and keep it and all other inputs fixed.
At baseline, \(\eta=-2+0.08(10)=-1.2\), giving \(p_0=1/(1+\exp(1.2))\approx0.2315\).
Under the alternative, \(\eta=-2+0.08(7.5)=-1.4\), giving \(p_1=1/(1+\exp(1.4))\approx0.1978\). Using the unrounded predictions, \(p_1-p_0\approx-0.03366\), a decrease of about 3.37 percentage points.
This comparison isolates the numerical impact of the exposure definition only under the stated compatibility assumption. If the coefficient must also be recalibrated to the alternative exposure scale, that recalibration is part of the interface change and should be described as such. Otherwise, the comparison may combine incompatible quantities rather than test a meaningful alternative.
Population alignment is another distinct interface question. A component may provide predictions for subgroups, while a downstream decision concerns a target population with different subgroup proportions. If subgroup predictions are \(p_k\) and target-population weights are \(w_k\), where \(\sum_k w_k=1\), the population-average probability is \(\sum_k w_kp_k\). Changing the weights tests the population-alignment assumption; changing subgroup predictions at the same time would make the comparison harder to interpret.
Worked Example: Aligning Subgroup Weights
Consider a hypothetical prediction with response probabilities of 0.40 in subgroup A and 0.20 in subgroup B. The baseline target population is 70% A and 30% B. An alternative population alignment uses weights of 40% A and 60% B. Assume the subgroup predictions remain fixed and the weights refer to mutually exclusive, exhaustive groups.
The baseline population-average probability is \(0.70(0.40)+0.30(0.20)=0.28+0.06=0.34\), or 34%.
The alternative is \(0.40(0.40)+0.60(0.20)=0.16+0.12=0.28\), or 28%. The change is \(0.28-0.34=-0.06\), a decrease of 6 percentage points. This difference reflects the altered subgroup composition, not a change in either subgroup’s predicted response.
Dependence Assumptions and a Justification
When linked components share uncertain quantities, their dependence can influence an integrated prediction. In particular, preserving only marginal distributions while assuming independence may produce a different expectation from retaining the covariance. This is not a claim that dependence always increases predictions: its direction depends on the quantities and the model relationship.
The justification follows from the definition \(\operatorname{Cov}(A,B)=E[(A-E[A])(B-E[B])]\). Expanding the product inside the expectation gives \(E[AB]-E[A]E[B]\), which rearranges to the stated result. This identity applies directly to a product outcome; for nonlinear predictions, the effect of dependence must generally be evaluated by propagating the alternative joint assumptions through the model.
Worked Example: Preserving Dependence in an Integrated Prediction
Suppose a hypothetical integrated model uses the product \(R=\theta E\), where \(\theta\) is an uncertain response coefficient measured in response units per exposure unit and \(E\) is exposure in exposure units. Let \(E[\theta]=0.5\), \(E[E]=8\), and \(\operatorname{Cov}(\theta,E)=0.4\) in the corresponding product units. Compare retaining this dependence with assuming independence while preserving the same marginal means.
With dependence retained, \(E[R]=E[\theta E]=E[\theta]E[E]+\operatorname{Cov}(\theta,E)=0.5(8)+0.4=4.4\) response units.
Under independence, the covariance is zero, so \(E[R]=0.5(8)=4.0\) response units. The independence alternative is 0.4 response units lower. This calculation isolates the covariance contribution for this product model. For a nonlinear endpoint prediction, the appropriate comparison is to run the integrated model under each joint assumption, rather than assume that the same difference carries over.
Designing and Checking the Sensitivity Analysis
A useful analysis makes the alternatives traceable to scientific or operational questions. For each interface, state the baseline choice, the alternative, why it is plausible, which other assumptions are held fixed, and which prediction is compared. The following sequence helps keep the comparison interpretable.
Name the endpoint, target population, dose or regimen, and prediction horizon.
State the baseline and alternative assumptions, including units and any required mapping or recalibration.
Change one assumption at a time for attribution; add joint scenarios when combined alternatives are scientifically relevant.
Report predictions, absolute changes, and suitable relative changes, then check implementation and interpretation against the model definitions.
Diagnostics include confirming that the intended interface actually changed, checking units and population weights, verifying that probabilities remain between 0 and 1, and examining whether the direction of change agrees with the model equations. Repeat computations or numerical approximations when small differences are close to computational tolerance. If scenarios produce abrupt, implausible changes, inspect transformations, boundary behavior, and whether a parameter or exposure definition was passed in the expected scale.
One-at-a-time comparisons are easy to explain but may miss interactions: two individually small changes can jointly have a large effect, or their effects can offset. Joint scenarios expose such behavior but do not by themselves identify which change drove it. A local derivative or small perturbation can summarize behavior near the baseline, but it may be misleading when the response is nonlinear or the alternative is far away. Broad scenario analysis can test a wider range of assumptions, but its conclusions depend on which scenarios were selected.
These approaches complement rather than replace one another. Unlike a hypothesis test, a sensitivity analysis does not provide a p-value or a probability that an assumption is true. Unlike model averaging, it does not combine alternatives into a single prediction with weights. It reports how specified choices affect the integrated result. Its limitations are important: conclusions are only as informative as the alternatives, the model structure, and the prediction target; unexamined interfaces may still matter.
Common Pitfall
A frequent error is to calculate a change from probabilities rounded for display. In the first example, subtracting displayed values can give a slightly different answer than subtracting the unrounded model outputs. Calculate differences using full computational precision, then round the difference for reporting. Also distinguish an absolute probability change from a relative percent change, and avoid interpreting either as evidence that the underlying assumption is true or false.
Check Your Understanding
Answer each question, then compare with the solutions.
- A baseline probability is 0.30 and an alternative probability is 0.27. What are the absolute change in probability and the decrease in percentage points?
- Why should a transferred coefficient not automatically be retained when the exposure definition changes?
- Subgroup probabilities are 0.50 and 0.10. What is the weighted population probability when the respective target weights are 0.25 and 0.75?
- If two uncertain quantities have covariance \(-0.2\), how does the expected product compare with the product of their means?
Solutions. (1) The change is \(0.27-0.30=-0.03\), a decrease of 3 percentage points. (2) The coefficient may be calibrated to the original exposure scale or definition; retaining it without checking compatibility can combine quantities that no longer have the intended meaning. (3) The weighted probability is \(0.25(0.50)+0.75(0.10)=0.125+0.075=0.20\), or 20%. (4) By \(E[AB]=E[A]E[B]+\operatorname{Cov}(A,B)\), the expected product is 0.2 below the product of the means, in the appropriate product units.