Why Compare Tipping Points Across Subgroups?
A sensitivity analysis can show how strongly conclusions depend on assumptions about missing outcomes. Repeating that analysis within prespecified subgroups can reveal whether the conclusion for one subgroup changes under a smaller assumed departure from the imputation model than the conclusion for another. That comparison is useful, but its meaning is limited: different tipping points describe different sensitivity of subgroup-specific conclusions, not, by themselves, a difference in treatment effects between subgroups.
The comparison is interpretable only after specifying the outcome, subgroup estimand, decision threshold, and delta path. In particular, the same numerical delta must represent the same kind of departure from the imputation model in each subgroup. Earlier material on standardizing delta parameters across outcomes is relevant to this requirement; here the focus is on calculations and interpretation across subgroups.
For illustration, let \(Q_g(\delta)\) be the treatment-effect estimate for subgroup \(g\) after imputations are adjusted using delta \(\delta\), and let \(SE_g(\delta)\) be its standard error. The estimate and standard error are obtained from the specified MICE analysis at that delta. A subgroup analysis must use the intended subgroup population and estimand; for example, an effect among subgroup members is not automatically the same estimand as an interaction contrast comparing that subgroup with another.
A one-dimensional subgroup comparison varies \(\delta_g\) for the subgroup being examined while holding the other sensitivity assumptions at their stated reference values. If separate subgroup-specific deltas are varied simultaneously, the result is instead a path through a multidimensional sensitivity analysis; it should not be reported as though it were a one-parameter tipping point.
Calculating a Crossing
Consider a two-sided, normal-approximation significance conclusion at level \(\alpha=0.05\). With a constant standard error \(s_g\) and critical value 1.96, the significance boundary is \(|Q_g(\delta)|=1.96s_g\). Suppose the estimate follows the linear path \(Q_g(\delta)=a_g-b_g\delta\), where \(a_g\) is the estimate at the reference delta and \(b_g>0\) means the estimate decreases as delta increases. If the subgroup is significant at the reference value, so that \(a_g>1.96s_g\), then the path reaches the positive significance boundary at
This expression follows by setting the estimate equal to the boundary and solving: \(a_g-b_g\delta=1.96s_g\), so \(b_g\delta=a_g-1.96s_g\). The assumptions matter. The starting estimate must be above the boundary, the path must actually reach it in the direction being considered, and the standard error and linear form must be adequate over the relevant range. If the standard error changes appreciably, solve \(|Q_g(\delta)|=1.96SE_g(\delta)\) using the computed sensitivity curve rather than this shortcut. A different inferential procedure may also use a different critical value.
Worked Example: Comparing Significance Tipping Points
Suppose hypothetical MICE sensitivity analyses estimate a treatment effect on systolic blood pressure in two subgroups. The delta is an additive shift in the imputed outcome, measured in mmHg, for the specified arm; positive delta values lower the estimated treatment benefit. Assume the reported curves are adequately represented by linear paths with constant standard errors. Subgroup A has \(Q_A(\delta)=4.8-0.8\delta\) mmHg and \(s_A=1.2\) mmHg. Subgroup B has \(Q_B(\delta)=3.0-0.5\delta\) mmHg and \(s_B=1.1\) mmHg. Find the first delta at which each loses two-sided significance under the 1.96 cutoff.
For A, the boundary estimate is \(1.96(1.2)=2.352\) mmHg. At the reference delta, \(4.8>2.352\), so A starts significant. The crossing is
At this value, \(Q_A(3.06)=4.8-0.8(3.06)=2.352\) mmHg, exactly the boundary.
For B, the boundary estimate is \(1.96(1.1)=2.156\) mmHg. B also starts significant because \(3.0>2.156\). Its crossing is
Thus the estimated significance conclusion for B reaches its boundary at a smaller delta than the conclusion for A, a difference of \(3.06-1.688=1.372\) mmHg along this common, interpretable delta scale. This indicates greater sensitivity for B on this particular path. It does not establish that the treatment effects differ between subgroups.
For a practical threshold, replace the significance boundary with the threshold that defines the decision. If a positive estimated benefit must be at least \(T\), a decreasing linear path \(a_g-b_g\delta\) reaches that threshold at \((a_g-T)/b_g\), provided \(a_g>T\), \(b_g>0\), and the path reaches \(T\). Be explicit about whether the decision changes at equality or only after the estimate passes below the threshold.
Worked Example: Comparing Practical-Threshold Tipping Points
In a hypothetical analysis, a larger positive treatment effect is considered clinically important if it is at least 4 units. The common delta is measured in the outcome’s units and has the same interpretation in both subgroups. Suppose subgroup C follows \(Q_C(\delta)=6.4-0.4\delta\), and subgroup D follows \(Q_D(\delta)=5.2-0.3\delta\). Assume these linear paths are reasonable over the ranges examined. Both start above the threshold, and both decrease as delta increases.
For C, solve \(6.4-0.4\delta=4\). Then \(0.4\delta=2.4\), so \(\delta_C^*=2.4/0.4=6.00\) outcome units. For D, solve \(5.2-0.3\delta=4\). Then \(0.3\delta=1.2\), so \(\delta_D^*=1.2/0.3=4.00\) outcome units.
The estimated benefit in D reaches the practical threshold at a delta 2.00 units smaller than in C. This comparison concerns the robustness of each subgroup estimate to the specified assumptions. It is not a test that the subgroup effects differ, and it depends on the threshold being substantively appropriate for both subgroups.
When the Sensitivity Curve Is Not a Straight Line
Subgroup estimates and standard errors need not vary linearly with delta. In that case, calculate or inspect the subgroup-specific curve and locate the first crossing of the chosen boundary in the stated direction. A grid can bracket a crossing; interpolation is an approximation whose adequacy depends on how smoothly the curve varies. A root-finding method can refine a bracket when the curve is sufficiently well behaved. The same crossing rule must be applied to each subgroup.
Worked Example: Bracketing and Interpolating Two Crossings
Suppose hypothetical sensitivity output gives subgroup A estimates of 2.8 at \(\delta=0\) and 1.8 at \(\delta=2\); subgroup B estimates are 2.4 at \(\delta=0\), 2.0 at \(\delta=2\), and 1.8 at \(\delta=3\). Assume the standard error is 1.00 in both groups throughout this range, and use the two-sided normal boundary 1.96. The estimates and delta are on a common outcome scale. These output values are hypothetical; linear interpolation is used only as an approximation between evaluated values.
For A, the estimate is above 1.96 at \(\delta=0\) and below it at \(\delta=2\), so the crossing is bracketed. The estimated slope over this interval is \((1.8-2.8)/(2-0)=-0.50\) per delta unit. Linear interpolation gives
For B, the estimate remains above the boundary at \(\delta=2\) but is below it at \(\delta=3\). The slope across that interval is \((1.8-2.0)/(3-2)=-0.20\). Thus
The output brackets A’s crossing between 0 and 2 and B’s between 2 and 3; the interpolated values are estimates, not exact crossings unless the curve is linear on those intervals. Under this approximation, A reaches the significance boundary earlier. If the curve bends materially between grid points, evaluate more points or use a suitable root-finding procedure instead of relying on this interpolation.
Interpret the Comparison, Not More Than It Says
A subgroup-specific tipping point answers a conditional question: how far must the specified missing-data assumption move along this path before the chosen conclusion for this subgroup reaches its boundary? Comparing two such values is meaningful when the estimands, boundary rules, and delta meanings are aligned. It is not a direct comparison of the treatment effects themselves.
In particular, a smaller tipping point in one subgroup does not demonstrate effect modification. Effect modification concerns whether treatment effects differ across subgroups, and requires a direct analysis of an interaction or of the contrast between subgroup-specific effects, with appropriate uncertainty. Even if one subgroup is significant at a given delta and another is not, that pattern alone does not show that their effects differ.
| Approach | Question answered | What it does not establish |
|---|---|---|
| Separate subgroup tipping points | At what delta does each subgroup’s stated conclusion reach its boundary? | Whether subgroup treatment effects differ |
| One common delta applied to all subgroups | How do subgroup conclusions behave under the same assumed shift? | How results change if subgroup-specific departures are allowed |
| Direct interaction or subgroup-effect contrast | Is there evidence that treatment effects differ between subgroups under the analysis model? | Robustness to unexamined missing-data assumptions |
Before reporting a comparison, check that each subgroup has enough observed information for stable imputation and estimation, and that the delta has the same substantive interpretation. Different missing fractions can produce different changes in estimates even under the same shift, so a tipping-point difference may reflect the data structure as well as the response to the assumption. Examine the subgroup sensitivity curves, including their standard errors, over the stated plausible delta range. Look for a crossing at the reference value, nonmonotone curves, wide or irregular changes, and crossings that occur only after implausibly large shifts. If there is no crossing in the examined range, report that fact rather than extrapolating an unverified tipping point.
Check Your Understanding
For each question, identify the relevant boundary and interpret the result in context. Solutions follow each question.
-
A subgroup estimate is \(Q(\delta)=5.0-0.5\delta\) with constant standard error \(s=1.0\). Under a two-sided normal 1.96 cutoff, find the significance tipping point and verify that significance is present at the reference delta.
Solution: At \(\delta=0\), \(Q=5.0>1.96(1.0)=1.96\), so the subgroup starts significant. Solving \(5.0-0.5\delta=1.96\) gives \(\delta^*=(5.0-1.96)/0.5=6.08\). Under the assumed linear path and constant standard error, the estimate reaches the positive significance boundary at 6.08 delta units.
-
A subgroup estimate is \(Q(\delta)=1.5-0.1\delta\), with \(s=1.0\). Does the positive significance-boundary formula give a first loss of significance as delta increases from zero?
Solution: No. At the reference value, \(1.5<1.96\), so the subgroup is already nonsignificant. The formula \((a-1.96s)/b\) gives \((1.5-1.96)/0.1=-4.6\), which is not a future crossing for increasing delta. There is no first loss of significance along this increasing-delta path.
-
Two subgroups have significance tipping points of 2 and 5 units on the same delta scale. What can be concluded?
Solution: Along the specified path and using the same boundary rule, the first subgroup’s significance conclusion reaches its boundary at a smaller delta. This indicates greater sensitivity of that conclusion on this path; it does not establish a difference in treatment effects between subgroups.
-
A subgroup estimate decreases from 3.0 at delta 0 to 2.2 at delta 2. Its standard error is constant at 1.0, and the significance boundary is 1.96. Use linear interpolation to estimate the crossing.
Solution: The slope is \((2.2-3.0)/2=-0.40\) per delta unit. The estimated crossing is \((3.0-1.96)/0.40=2.60\). This is an interpolation-based estimate; the curve must be evaluated further if linearity between the grid points is doubtful.
-
Why can different missing fractions complicate a comparison made using the same numerical delta?
Solution: The same assumed shift may affect different numbers or proportions of imputed outcomes in each subgroup. Consequently, the estimates may move by different amounts even when the delta has a common meaning. The resulting tipping-point difference describes sensitivity in the observed subgroup analyses, not solely a contrast in treatment effects.