Detecting Interactions Between Decision Assumptions

Why examine the joint effect?

Two assumptions can each change a model-informed decision prediction when varied alone. Their joint effect may nevertheless differ from the sum of those separate changes. For example, two assumptions about adherence and recruitment might each increase predicted development value, while the model predicts a larger increase when both apply. Detecting that departure matters because one-at-a-time sensitivity results do not describe how assumptions combine.

The key comparison is between the prediction under both assumptions and an additive benchmark: the prediction expected if each assumption contributed the same change when the other was present as when it was absent. This is a comparison of scenario predictions, not a test of whether an assumption is true.

Set up a valid four-scenario comparison

Before calculating the contrast, define the two assumptions, their reference and alternative levels, and the prediction \(R\) being compared. \(R\) might be expected net benefit in dollars, probability of meeting a development target, or a predicted clinical outcome. The four scenario predictions must use the same definition of \(R\), the same model and decision rules, and the same settings for other inputs. Otherwise, differences may reflect changes beyond the two assumptions of interest.

The scenarios should be substantively interpretable and sufficiently plausible to support the comparison. In particular, the combined scenario must be evaluated as a joint scenario in the model; it should not simply be inferred from the two separate scenarios. If the combined assumptions are incompatible, or their joint consequences are not represented by the model, the contrast may have little decision value.

The contrast is scale-specific. An interaction of zero means additivity on the chosen scale, not absence of every possible interaction. For example, predictions may be additive in dollars but not in log dollars. Probability predictions are often naturally compared by differences in percentage points, but a probability scale is bounded between zero and one; the additive benchmark should be checked for sensible interpretation.

Why the contrast isolates departure from additivity

The separate changes from the reference scenario are \(R_{10}-R_{00}\) for A and \(R_{01}-R_{00}\) for B. Under an additive benchmark, the predicted joint change from \(R_{00}\) is the sum of these changes. Adding them to the reference prediction gives \(R_{00}+(R_{10}-R_{00})+(R_{01}-R_{00})\), which simplifies to \(R_{10}+R_{01}-R_{00}\). Subtracting this benchmark from the observed joint prediction gives the interaction contrast.

Thus, \(I_{AB}=0\) means the joint prediction matches the additive benchmark. A positive value means the joint prediction is higher than the benchmark; a negative value means it is lower. “Higher” is not automatically “better”: the decision metric may be an undesirable outcome, such as toxicity risk, for which a lower prediction could be preferred. State the result in the metric’s units and explain its decision meaning.

Worked examples on different decision scales

Diagnostics, alternatives, and limitations

A two-way sensitivity analysis displays predictions across combinations of two assumptions. The interaction contrast adds a focused numerical summary of how one joint scenario compares with additivity, but it does not replace inspection of the full grid. When assumptions have several levels, calculate the contrast for clearly specified combinations or examine the complete grid for changing patterns. A single four-cell contrast summarizes only the chosen levels.

One-way sensitivity analysis changes one assumption at a time and can show each separate effect, but it cannot reveal whether their combination departs from additivity. The additive contrast uses those one-at-a-time predictions as its benchmark and requires the joint prediction as well. A multiplicative benchmark is an alternative when ratios are scientifically meaningful and predictions are positive: the comparison may be made on a log scale, where additivity corresponds to multiplicative combination on the original scale. That is a different question, so the scale should be chosen before interpreting the result.

The interaction contrast is descriptive for the specified model, scenarios, and prediction metric. It is not a p-value, a probability that an interaction exists, or proof of a biological or causal interaction. If the predictions include uncertainty, report or otherwise examine that uncertainty separately; a point contrast alone does not convey its precision.

Several checks help identify a misleading result:

• Check scenario consistency. Confirm that the reference and alternative levels are defined identically in all four cells and that other assumptions were held fixed.
• Check the joint prediction. Make sure the model actually represents both assumptions together and that the combination is plausible, rather than relying on an unsupported extrapolation.
• Check the metric and units. A percentage-point contrast is not a relative percent change. A positive contrast is not inherently favorable.
• Check the threshold separately. A nonzero contrast can leave the decision unchanged, while a zero contrast can still accompany a threshold crossing. The contrast and the decision rule answer different questions.
• Check sensitivity to the chosen levels. An interaction can be large for one pair of scenario levels and small for another. Do not generalize a local contrast to every possible value of the assumptions.

A common mistake is to say, “A adds \(\$3\) million and B adds \(\$2\) million, so together they must add \(\$5\) million.” That reasoning assumes the joint scenario equals the additive benchmark. The correction is to calculate the model’s joint prediction, compare it with \(R_{10}+R_{01}-R_{00}\), and report the difference as the interaction contrast. Another mistake is to call any change in a go/no-go outcome an interaction: a threshold can make a decision flip even when the prediction itself is additive. First compare predictions on a stated scale; then apply the decision rule.