Arm-Specific Tipping Thresholds for Treatment Comparisons

Why Separate the Two Arms?

A sensitivity analysis for a treatment comparison can allow missing outcomes in the treatment arm and control arm to depart from their respective reference predictions by different amounts. This distinction matters: the same shift among missing outcomes need not have the same effect on the comparison in each arm, because the arms can have different missing fractions. A treatment-only threshold and a control-only threshold are therefore useful summaries, but neither tells the whole story. The treatment comparison depends on their joint values.

Consider a continuous outcome for which higher values are better, and define the treatment effect as the treatment-arm mean minus the control-arm mean. Use a MAR analysis as the reference: for each arm, its reference mean is the mean that the analysis would estimate under its MAR assumptions. An arm-specific delta is an additive shift, in outcome units, applied to missing outcomes relative to that reference. A negative delta lowers the assumed mean of missing outcomes; a positive delta raises it.

Definition: Let $r_T$ and $r_C$ be the fractions of outcomes missing in the treatment and control arms. Let $\hat{\mu}_T^{MAR}$ and $\hat{\mu}_C^{MAR}$ be the arm means from the MAR reference analysis, and let $\delta_T$ and $\delta_C$ be additive shifts for missing outcomes in the two arms. In a simple arm-mean pattern-mixture analysis, the adjusted contrast is $$ \hat{\Delta}(\delta_T,\delta_C) = \left(\hat{\mu}_T^{MAR}-\hat{\mu}_C^{MAR}\right) +r_T\delta_T-r_C\delta_C. $$ Write $\hat{\Delta}_{MAR}=\hat{\mu}_T^{MAR}-\hat{\mu}_C^{MAR}$. The arm-specific sensitivity shifts change the contrast by $r_T\delta_T-r_C\delta_C$, not by $\delta_T-\delta_C$ alone.

The fractions appear because a delta changes only the assumed outcomes among participants with missing outcomes. For example, if 20% of an arm’s outcomes are missing, a 1-unit shift in those missing outcomes changes that arm’s overall mean by 0.2 units. In more complex adjusted analyses, the mapping from delta parameters to the estimated contrast may not equal these simple missing-fraction weights. The formula above applies to the stated arm-mean setup; use the actual model’s contrast function when its structure gives different weights.

Assumptions and the Tipping Boundary

Before calculating thresholds, specify the analysis and decision rule. The examples below use a two-sided 95% confidence interval and ask whether its lower endpoint remains above zero, supporting a positive treatment effect. For the boundary calculations, the standard error is held fixed at its MAR-reference value. Thus the lower endpoint is $\hat{\Delta}(\delta_T,\delta_C)-1.96SE$. A different confidence level, a finite-degrees-of-freedom critical value, or a rule based on effect direction alone gives a different boundary.

The simple formula assumes the missing fractions are treated as fixed, each missing outcome in an arm receives the same additive shift relative to its MAR reference, and the contrast is the difference between the adjusted arm means. Holding $SE$ fixed is an additional sensitivity-analysis convention, not a mathematical consequence of the mean adjustment. If the analysis recalculates the standard error for each delta pair, the confidence-limit boundary must be found using that recalculated standard error.

Let $c$ denote the critical value used for the confidence interval. A positive effect is statistically significant by the specified lower-limit rule when $\hat{\Delta}(\delta_T,\delta_C)-cSE>0$. The boundary is equality; on the side where the lower limit is below zero, the rule no longer supports a positive effect. Solving for the arm-specific shifts gives a line in the $(\delta_T,\delta_C)$ plane.

\hat{\Delta}_{MAR}+r_T\delta_T-r_C\delta_C=cSE \qquad\Longleftrightarrow\qquad r_T\delta_T-r_C\delta_C=cSE-\hat{\Delta}_{MAR}.

This boundary is a joint condition. A treatment-only threshold sets $\delta_C=0$; a control-only threshold sets $\delta_T=0$. Those are axis intercepts of the boundary, not two independent descriptions of every possible joint tipping combination. The derivation follows directly from the decision rule: substitute the adjusted contrast into the lower confidence limit, set that limit equal to zero, and rearrange. This establishes the boundary without requiring a separate approximation beyond the specified fixed standard error and critical value.

Worked Example: Treatment-Only and Control-Only Thresholds

Suppose a hypothetical trial has $\hat{\mu}_T^{MAR}=8.0$ points and $\hat{\mu}_C^{MAR}=6.5$ points on a scale where higher is better. The treatment-arm missing fraction is $r_T=0.20$, the control-arm missing fraction is $r_C=0.30$, and the MAR contrast has $SE=0.55$ points. Assume the fixed-standard-error 95% lower-limit rule described above.

The MAR contrast is $8.0-6.5=1.5$ points. Its lower confidence limit is $1.5-1.96(0.55)=1.5-1.078=0.422$ points, so the reference analysis supports a positive effect by this rule. The tipping boundary requires the adjusted contrast to equal $1.078$ points:

1.5+0.20\delta_T-0.30\delta_C=1.96(0.55)=1.078.

For the treatment-only threshold, set $\delta_C=0$. Then $0.20\delta_T=1.078-1.5=-0.422$, so $\delta_T=-0.422/0.20=-2.11$ points. Under this one-arm scenario, the assumed mean among missing treatment participants must be 2.11 points below its MAR reference for the lower confidence limit to reach zero.

For the control-only threshold, set $\delta_T=0$. Then $-0.30\delta_C=-0.422$, so $\delta_C=0.422/0.30=1.407$ points. Here, missing control outcomes must be shifted 1.407 points above their MAR reference to reach the same boundary. The signs differ because lowering the treatment mean and raising the control mean both reduce a treatment-minus-control contrast.

How the Two Thresholds Combine

The two axis thresholds in the example are not a requirement that both shifts occur at once. They are alternative one-arm scenarios. When both deltas are nonzero, their weighted contributions add. In this example, every pair on $0.20\delta_T-0.30\delta_C=-0.422$ lies on the confidence-limit boundary. Pairs on the side with a smaller adjusted contrast fail the positive lower-limit rule; pairs on the other side retain it.

Worked Example: Evaluating a Joint Pair of Shifts

Use the same hypothetical trial. Suppose the assumed missing treatment outcomes are shifted down by 1.0 point, while missing control outcomes are shifted up by 0.74 points: $\delta_T=-1.0$ and $\delta_C=0.74$.

The adjusted contrast is $1.5+0.20(-1.0)-0.30(0.74)=1.5-0.20-0.222=1.078$ points. The lower 95% confidence limit is $1.078-1.96(0.55)=1.078-1.078=0$ points. This pair is exactly on the tipping boundary under the stated calculation.

A slightly more adverse pair, such as $\delta_T=-1.2$ and $\delta_C=0.62$, gives $1.5+0.20(-1.2)-0.30(0.62)=1.5-0.24-0.186=1.074$ points. Its lower limit is $1.074-1.078=-0.004$ points, just below zero. The rule no longer supports a positive effect at the 5% two-sided level. The small difference illustrates why rounding near a boundary should be reported carefully.

The boundary can also be written as a control delta for each treatment delta: $\delta_C=(\hat{\Delta}_{MAR}+r_T\delta_T-cSE)/r_C$, provided $r_C>0$. This makes clear that there are infinitely many joint pairs on a straight boundary in this simple model, rather than one unique pair of thresholds. The line’s slope is $r_T/r_C$; its intercepts depend on the MAR contrast and the confidence-limit criterion.

Effect Direction Is a Different Decision Rule

A tipping point for loss of statistical significance is not necessarily a tipping point for reversal of the estimated effect. For effect direction, the boundary is where the adjusted contrast itself equals zero. With the same hypothetical MAR contrast and missing fractions, this condition is $1.5+0.20\delta_T-0.30\delta_C=0$. It gives treatment-only $\delta_T=-7.5$ points and control-only $\delta_C=5.0$ points. These differ from $-2.11$ and $1.407$, the significance thresholds, because the confidence-limit rule reaches zero while the estimated contrast is still positive.

Worked Example: A Common Shift Can Move the Contrast Unexpectedly

Suppose the same trial instead uses a common delta of $-2.0$ points in both arms: $\delta_T=\delta_C=-2.0$. This means missing outcomes are assumed to be 2.0 points below their MAR references in both arms. The adjusted contrast is $1.5+0.20(-2.0)-0.30(-2.0)=1.5-0.4+0.6=1.7$ points.

Although the assumed outcomes are lower in both arms, the treatment contrast increases by 0.2 points. The reason is that the control arm has the larger missing fraction: the common downward shift lowers its overall mean more. This calculation is consistent with the formula and shows why a common delta should not automatically be described as adverse to the treatment comparison.

Diagnostics, Alternatives, and Limitations

Several checks help prevent interpretation errors. Confirm that the delta units match the outcome units and that the sign convention is explicit. Verify the missing fractions and the reference contrast, then substitute each proposed pair into the contrast formula before classifying it relative to the boundary. Check which side of the boundary satisfies the chosen decision rule; a contour or table can otherwise be labeled in the wrong direction. If the standard error is held fixed, state that convention, especially when examining large shifts that could materially change the analysis.

A one-parameter analysis, such as imposing the same delta in both arms, is simpler to display but examines only a line through the two-dimensional set of possible arm-specific shifts. It can miss combinations where the arms move in opposite directions or at different rates. A two-parameter boundary preserves that distinction, while a full contour display can show the adjusted decision over a grid of pairs. These approaches use the same underlying sensitivity assumptions when their model and decision rule match; they differ in how much of the joint parameter space they reveal.

Separate thresholds are summaries, not estimates of what missing outcomes actually were. They do not establish that a particular delta is plausible, and they do not account automatically for uncertainty in the estimated boundary. The results also depend on the chosen reference analysis, the additive-shift form, the outcome scale, the missing fractions, and the decision rule. If there are no missing outcomes in an arm, its delta has no effect on this arm-mean contrast, so a finite threshold for that arm cannot be obtained from this formula. Finally, a numerical threshold close to zero can be sensitive to rounding; retain sufficient precision in calculations and describe the result in the outcome’s units.

Common pitfall: It is incorrect to subtract $\delta_T-\delta_C$ from the treatment contrast without accounting for how many outcomes are missing in each arm. The arm-specific shifts apply to missing outcomes, so their effects on the overall contrast are weighted by $r_T$ and $r_C$. A second error is to treat the two one-arm thresholds as if they uniquely specify all joint tipping pairs. They are only the boundary’s intercepts when the other arm’s delta is set to zero.

Check Your Understanding

Use the hypothetical values $\hat{\Delta}_{MAR}=1.5$ points, $r_T=0.20$, and $r_C=0.30$. For questions involving significance, use $SE=0.55$ points, a 1.96 critical value, and a fixed standard error.

Calculate the adjusted contrast when $\delta_T=-2$ points and $\delta_C=1$ point. Does it meet the positive 95% lower-limit rule?
With $\delta_T=0$, calculate the control-only shift at the significance tipping boundary.
If both arms use $\delta=-1$ point, calculate the adjusted contrast. Explain why it is not $1.5$ points.
Name two assumptions or conventions that must be checked before interpreting a calculated threshold.

Solutions. (1) The adjusted contrast is $1.5+0.20(-2)-0.30(1)=1.5-0.4-0.3=0.8$ points. Its lower limit is $0.8-1.078=-0.278$ points, so it does not meet the rule. (2) Set $1.5-0.30\delta_C=1.078$, giving $\delta_C=0.422/0.30=1.407$ points. (3) The adjusted contrast is $1.5+0.20(-1)-0.30(-1)=1.5-0.2+0.3=1.6$ points. The common shift does not cancel because the missing fractions differ. (4) Examples include checking that the deltas are additive shifts in outcome units relative to the MAR reference, confirming the arm-specific missing fractions and contrast definition, and stating whether the standard error is fixed or recalculated.