Minimum-Effect Thresholds For Continuing Subgroups

Why a Threshold Adds to an Interim Decision

An interim estimate can suggest that a treatment helps a biomarker-defined subgroup and still fall short of the effect that would make continued enrollment worthwhile. A conventional test against no effect asks whether the data are inconsistent with zero benefit. An enrichment decision often asks a different question: is the effect large enough to justify continuing this subgroup?

That second question needs a clinically interpretable benchmark, stated in the units of the treatment effect. This benchmark should be defined before the interim data are examined. It does not replace a prespecified enrichment decision rule; it makes the clinical target within that rule explicit. The threshold is not, by itself, a significance level or proof of efficacy.

Definition: For a prespecified subgroup \(S\), let \(\theta_S\) denote the treatment effect on a prespecified estimand and scale, oriented so that larger values mean greater benefit. A minimum-effect threshold, written \(\delta_{\mathrm{clin}}\), is the smallest benefit on that scale judged sufficient to make continuing or enriching to \(S\) worthwhile. Effects below the threshold may still be beneficial, but are not considered large enough for that decision.

For example, if \(\theta_S\) is the mean reduction in a symptom score for treatment relative to control, measured in score points, a threshold of 3 means that a 3-point average reduction is the minimum benefit judged worthwhile. If the natural effect measure has the opposite direction, such as a difference in symptom scores where lower values are better, define the benefit contrast with the sign reversed before applying this framework.

The estimand must specify the population, treatment conditions, outcome, and time point. A threshold of 3 points for a subgroup at 12 weeks is not automatically meaningful for the all-comer population or for a 24-week outcome. The threshold should also be justified clinically: it may reflect patient-valued improvement, treatment burden, harms, available alternatives, and the consequences of continuing recruitment. It is not necessarily identical to an individual-level minimum clinically important difference; the trial estimand may be a population-average effect.

Two Different Questions: Benefit and Worthwhile Benefit

Let \(\hat{\theta}_S\) be the interim estimate of \(\theta_S\), with standard error \(SE\). A test of no benefit considers \(H_0:\theta_S\leq 0\) against \(H_a:\theta_S>0\). A test of whether the effect exceeds the minimum worthwhile level instead considers \(H_0:\theta_S\leq\delta_{\mathrm{clin}}\) against \(H_a:\theta_S>\delta_{\mathrm{clin}}\). These hypotheses address distinct scientific claims. A small p-value for the first does not establish the second.

A simple decision rule might continue subgroup \(S\) if \(\hat{\theta}_S\geq\delta_{\mathrm{clin}}\). It is easy to explain, but ignores precision. A more evidence-demanding rule requires a one-sided lower confidence bound to exceed the threshold. Under an approximately normal estimator with a valid standard error, a lower bound with one-sided confidence level \(1-\gamma\) is

L=\hat{\theta}_S-z_{1-\gamma}SE,

where \(z_{1-\gamma}\) is the \(1-\gamma\) quantile of the standard normal distribution. The rule continues the subgroup only if \(L>\delta_{\mathrm{clin}}\). Equivalently, the standardized statistic for testing the threshold is

Z_{\delta}=\frac{\hat{\theta}_S-\delta_{\mathrm{clin}}}{SE}; \qquad \text{continue if } Z_{\delta}>z_{1-\gamma}.

The value of \(\gamma\) is part of the decision design and is not automatically the trial’s final significance level. A one-sided 80% lower bound, for instance, is a relatively permissive interim evidence criterion; it is not a 5% confirmatory test. If there are repeated interim looks or multiple candidate populations, the decision rule’s operating characteristics need to be assessed for that design. A nominal bound at one look does not automatically control the probability of an erroneous continuation across multiple looks or populations.

Interpretation: Requiring \(L>\delta_{\mathrm{clin}}\) means the chosen lower confidence bound is above the minimum worthwhile effect. It does not mean there is a \(1-\gamma\) probability that the fixed treatment effect exceeds the threshold. Under repeated sampling and the model assumptions, the bound procedure has its stated coverage.

Worked Threshold Decisions

Worked Example: Significant Benefit, but Not Evidence Above the Threshold

Hypothetical data. At an interim analysis in a biomarker-positive subgroup, the estimated mean symptom-score reduction for treatment versus control is \(\hat{\theta}_S=4.2\) points, with \(SE=1.8\) points. Larger values represent greater symptom reduction. The prespecified minimum worthwhile reduction is \(\delta_{\mathrm{clin}}=3.0\) points.

Assume randomized treatment groups, an unbiased estimate of the prespecified subgroup estimand, and an approximately normal sampling distribution with the reported standard error. Those assumptions support the following normal-approximation calculations; the hypothetical analysis has no evidence here of a gross model or data-quality problem.

First compare the estimate with zero. The statistic is \(4.2/1.8=2.33\), giving a one-sided normal-approximation p-value of about \(0.0098\). This is evidence against no benefit under the stated model.

Now compare the estimate with the clinically meaningful threshold:

Z_{\delta}=\frac{4.2-3.0}{1.8} =\frac{1.2}{1.8} =0.667.

The one-sided p-value for \(H_0:\theta_S\leq3.0\) is approximately \(0.252\). Thus the data are compatible with a positive effect, but do not provide strong evidence that the effect exceeds 3.0 points. A point-estimate rule would continue because \(4.2\geq3.0\); a rule requiring a one-sided 80% lower bound above 3.0 would not, since \(4.2-0.8416(1.8)=2.69\) points. The decision depends on the prespecified rule, not on relabelling the significant test against zero as proof of a worthwhile effect.

Worked Example: A Lower Bound Clears the Threshold

Hypothetical data. In another interim analysis, the estimated subgroup benefit is 5.1 points with \(SE=1.2\) points. The minimum worthwhile effect remains 3.0 points. Use a prespecified one-sided 80% lower-bound rule. Assume independent randomized groups, the same well-defined score-reduction estimand, and an adequate normal approximation for the estimate.

The standard normal quantile for a one-sided 80% lower bound is \(z_{0.80}=0.8416\). Therefore,

L=5.1-0.8416(1.2) =5.1-1.010 =4.09\text{ points}.

Because \(4.09>3.0\), the lower-bound rule supports continuing this subgroup. In context, the chosen one-sided lower confidence bound for mean symptom-score reduction is 4.09 points, above the 3.0-point minimum worthwhile effect. This is evidence under the specified rule, not a guarantee that the true effect exceeds 3.0 points.

Worked Example: The Point Estimate Passes, but Its Uncertainty Matters

Hypothetical data. A third subgroup has an interim estimated benefit of 2.7 points with \(SE=0.9\) points, and the prespecified threshold is 2.5 points. Assume the same estimand and normal-approximation conditions as above. The point estimate is above the threshold, so a rule based only on \(\hat{\theta}_S\geq\delta_{\mathrm{clin}}\) would continue.

Under the one-sided 80% lower-bound rule, the calculation is

L=2.7-0.8416(0.9) =2.7-0.7574 =1.94\text{ points}.

Since \(1.94\) is less than \(2.5\), the lower-bound rule does not support continuing on the basis of this interim evidence. This result does not show that the treatment has no benefit: the point estimate is positive and exceeds the threshold. It shows that the estimate is not sufficiently precise to satisfy this particular evidence rule. Whether the protocol stops, continues, or allows another prespecified action depends on the complete decision plan.

Why the Rule Behaves This Way

The distinction between a point-estimate threshold and an evidence threshold follows directly from the sampling distribution. Suppose, for justification, that \(\hat{\theta}_S\) is approximately normal with mean \(\theta_S\) and known or adequately estimated standard error \(SE\). Under the point-estimate rule, continuation occurs when \(\hat{\theta}_S\geq\delta_{\mathrm{clin}}\). If the true effect is exactly at the threshold, then

P(\hat{\theta}_S\geq\delta_{\mathrm{clin}}\mid \theta_S=\delta_{\mathrm{clin}}) =P\left(\frac{\hat{\theta}_S-\theta_S}{SE}\geq0\right) =0.5.

Thus, at the boundary effect, a point-estimate rule continues in about half of repeated samples under this model. It is a transparent screening rule, not a confidence statement.

For the lower-bound rule, continuation requires \(\hat{\theta}_S-z_{1-\gamma}SE>\delta_{\mathrm{clin}}\). At the boundary \(\theta_S=\delta_{\mathrm{clin}}\), this is equivalent to

\frac{\hat{\theta}_S-\theta_S}{SE}>z_{1-\gamma}, \qquad P(\text{continue}\mid\theta_S=\delta_{\mathrm{clin}})=\gamma.

This gives a useful interpretation of the evidence stringency in a single-look normal model: at the threshold boundary, the chance of passing the lower-bound rule is \(\gamma\). The derivation depends on the assumed sampling distribution and a fixed, valid standard error. With adaptive interim monitoring, correlated looks, or a data-dependent analysis, the actual operating characteristics may differ and should be evaluated for the full design.

Choosing and Checking a Threshold

A defensible threshold is specific enough that another analyst can apply it without interpreting the interim results. Before enrollment, document:

The target estimand and subgroup: define the population, outcome, follow-up time, treatment contrast, and direction of benefit.
The threshold and units: state a numerical value in the same units as the estimand, including how the value was justified clinically.
The decision criterion: say whether continuation depends on the point estimate, a confidence bound, a posterior probability, or another prespecified quantity, and state its cutoff.
The consequences of the decision: specify what “continue” and “do not continue” mean operationally, without assuming that failure to pass proves no benefit.
Uncertainty and repeat looks: assess how the rule behaves under plausible true effects, standard errors, subgroup prevalence, and the planned interim schedule.

Sensitivity analysis is especially useful when the clinical threshold is not known precisely. Evaluate the consequences of nearby plausible thresholds rather than selecting the value that makes the observed interim estimate pass. Also check that the interim estimate and its standard error refer to the same prespecified estimand and subgroup. Small subgroup counts, missing outcomes, model instability, or changes in assay classification can make the nominal standard error misleading; investigate these issues before treating a calculated bound as reliable.

A rule based on a lower confidence bound is more conservative than a point-estimate rule at the same threshold, but may be less likely to continue a truly worthwhile subgroup when interim information is sparse. A Bayesian decision rule can instead use a posterior probability or expected utility, but its conclusion depends on the prior and decision consequences. Conditional-power rules ask about the chance of meeting a future statistical target under specified assumptions; they need not encode a clinically meaningful effect unless that target is explicitly tied to one. These approaches answer related but not interchangeable questions. Whatever approach is chosen, the minimum-effect threshold should remain clinically justified and prespecified.

Common Pitfalls and Limitations

Confusing statistical significance with clinical importance. A small p-value against zero does not show that the effect exceeds \(\delta_{\mathrm{clin}}\). Test or bound against the threshold when that is the decision question.
Treating the threshold as a universal constant. Its meaning depends on the estimand, outcome scale, time point, population, and treatment context. State these alongside the number.
Ignoring precision. A point estimate above the threshold may be unstable. If uncertainty matters to the decision, use a prespecified uncertainty-based rule rather than interpreting the estimate alone.
Calling failure to pass evidence of no effect. A rule can fail because the effect is small, because the estimate is imprecise, or both. Report the estimate and its uncertainty, not just the decision label.
Changing the threshold after seeing interim results. This makes the decision rule data-dependent and can distort its operating characteristics. If clinical knowledge changes, document the rationale and assess the revised design rather than retrofitting the threshold.

A threshold-based continuation decision is a design choice under uncertainty, not a final efficacy claim. It can make enrichment decisions more clinically aligned, but does not by itself resolve multiplicity, selection bias in later estimates, or confirmatory error control. Those issues require attention in the full adaptive design.

Check Your Understanding

Answer each question using the threshold and decision-rule distinctions developed above. Solutions follow the questions.

An interim estimate of a subgroup benefit is 3.6 units with \(SE=1.0\); the minimum worthwhile effect is 2.0 units. Calculate the statistic for testing whether the effect exceeds the threshold. What does this statistic say, and what does it not say?
For an estimate of 4.0 units with \(SE=0.8\), calculate a one-sided 80% lower bound. Does it exceed a 3.0-unit threshold?
Why can a subgroup pass a test against zero benefit but fail a test against a positive minimum-effect threshold?
At a true effect equal to the threshold, what is the single-look continuation probability for (a) the point-estimate rule and (b) a one-sided lower-bound rule with \(\gamma=0.20\), under the normal model?

Solutions. (1) \(Z_{\delta}=(3.6-2.0)/1.0=1.60\). This is the standardized distance above the minimum-effect threshold; a one-sided normal p-value is about 0.0548. It is not a test of whether the effect is nonzero, nor is it by itself a continuation decision unless the protocol specifies the relevant cutoff. (2) \(L=4.0-0.8416(0.8)=4.0-0.6733=3.33\) units. Since \(3.33>3.0\), it passes this lower-bound criterion. (3) The threshold test places the null boundary at a positive effect, so the estimate must be sufficiently far above that larger boundary relative to its standard error. An estimate can be far enough above zero but not far enough above the clinically meaningful threshold. (4) For the point-estimate rule, the probability is 0.5; for the lower-bound rule it is \(\gamma=0.20\), by the normal-model derivation above. These are single-look probabilities under the stated assumptions, not guarantees for a multi-look adaptive design.