Optimizing Enrichment Thresholds For Power And Coverage

Why Threshold Choice Is a Tradeoff

An enrichment rule uses interim evidence to decide whether recruitment should focus on a candidate responsive population. A stricter rule can reduce the chance of enriching when the evidence is weak, but it can also miss a truly responsive population. Conversely, a permissive rule may preserve access to a broader population while selecting the responsive group more often. Choosing a threshold therefore requires specifying both what the trial hopes to detect and what it must avoid excluding.

Here, coverage means the probability that the rule does not exclude a specified population in a scenario where that population has clinically meaningful benefit. It is an operating characteristic of the selection rule, not the coverage probability of a confidence interval. Power refers to the probability of a successful confirmatory result through the enriched path in a responsive-population scenario.

A Threshold Model and Its Assumptions

Definition: Let $S$ be a standardized interim evidence score, with larger values favoring enrichment. The rule enriches the candidate responsive population if $S>c$, where $c$ is a finite threshold. Otherwise, it does not exclude the population protected by the rule. In a protected-population scenario $B$, define coverage as $C_B(c)=P(S\le c\mid B)$. In a responsive-population scenario $R$, define selection-stage power as $\operatorname{Power}_R(c)=q_RP(S>c\mid R)$, where $q_R$ is the probability of a successful confirmatory result conditional on entering the enriched path.

For the calculations below, assume that the evidence score has unit variance and is normally distributed under each scenario:

S\mid B\sim N(\mu_B,1),\qquad S\mid R\sim N(\mu_R,1).

The means $\mu_B$ and $\mu_R$ describe the expected evidence in their respective scenarios. The standard normal cumulative distribution function is denoted by $\Phi$. The normal model gives

C_B(c)=\Phi(c-\mu_B),\qquad \operatorname{Power}_R(c)=q_R\{1-\Phi(c-\mu_R)\}.

These expressions rely on a calibrated score, specified scenario distributions, a threshold fixed for the decision rule, and a conditional confirmatory success probability $q_R$ that does not change with $c$. The product form for power is appropriate when $q_R$ represents the confirmatory success probability after selection and is constant across the thresholds being compared. If the confirmatory design or its power changes with the selected rule, that dependence must be included rather than treated as a constant. Likewise, the normal distributions are planning assumptions to be checked against the intended design, not facts guaranteed by the use of a score.

This two-scenario model describes the tradeoff for a specified protected scenario and a specified responsive scenario. If several clinically meaningful scenarios must be protected, each needs its own coverage requirement. The evidence score can have different means in these scenarios; they are alternative data-generating conditions, not simultaneous values of $S$.

The Constrained Optimum

Suppose the design requires at least $C_{\min}$ coverage in scenario $B$, with $0<C_{\min}<1$, and seeks the greatest possible $\operatorname{Power}_R(c)$ among finite thresholds. Coverage increases with $c$, while power decreases with $c$. Thus, if $q_R>0$, the best feasible threshold is the smallest one that meets the coverage requirement.

\begin{aligned} \Phi(c-\mu_B)&\ge C_{\min},\\ c-\mu_B&\ge \Phi^{-1}(C_{\min}),\\ c&\ge \mu_B+\Phi^{-1}(C_{\min}). \end{aligned}

Result: For finite thresholds, $0<C_{\min}<1$, $q_R>0$, and the stated normal model, the power-maximizing threshold subject to $C_B(c)\ge C_{\min}$ is $$ c^*=\mu_B+\Phi^{-1}(C_{\min}). $$ At this threshold, coverage equals $C_{\min}$. The result assumes the only constraint is this coverage requirement and $q_R$ is constant across thresholds.

The optimality follows from monotonicity. Since $\Phi$ is strictly increasing, $C_B(c)$ is strictly increasing in $c$. Since $1-\Phi(c-\mu_R)$ is strictly decreasing in $c$, increasing the threshold beyond the smallest feasible value reduces power. The optimizer therefore lies exactly at the coverage boundary.

The boundary cases matter. If $C_{\min}=1$, no finite $c$ satisfies the constraint because $\Phi(c-\mu_B)<1$ for every finite $c$. Coverage approaches 1 only as $c$ tends to infinity, while power approaches 0. If $C_{\min}>1$, the requirement is infeasible. If $C_{\min}\le 0$, the coverage constraint is vacuous for finite $c$; with only that constraint, power has no finite maximizer, because it increases as $c$ decreases and approaches $q_R$ as $c$ tends to negative infinity.

Worked Examples

Worked Example: Meeting a 90% Coverage Requirement

Suppose the interim score has mean $\mu_B=0.40$ in the protected scenario and mean $\mu_R=1.30$ in the responsive scenario. Let $q_R=0.80$, and require $C_{\min}=0.90$. The unit-variance normal model is the planning assumption in both scenarios.

The 90th percentile of the standard normal distribution is $\Phi^{-1}(0.90)=1.2816$. Therefore, the smallest threshold meeting the coverage requirement is

$$ c^*=0.40+1.2816=1.6816. $$

At this threshold, $c^*-\mu_B=1.2816$, so coverage is $\Phi(1.2816)=0.9000$. The responsive-scenario standardized threshold is $1.6816-1.30=0.3816$. Thus,

\operatorname{Power}_R(c^*) =0.80\{1-\Phi(0.3816)\} \approx 0.80(0.3513) =0.2810.

The modeled probability of a successful confirmatory result through the enriched path is about 28.1%, with 90.0% coverage in scenario $B$. For comparison, a threshold $c=1.00$ would give coverage $\Phi(0.60)=0.7257$ and power $0.80\{1-\Phi(-0.30)\}\approx0.4943$. That lower threshold has higher power in scenario $R$, but it fails the required coverage.

Worked Example: The Cost of Raising the Coverage Target

Now let $\mu_B=0.70$, $\mu_R=1.80$, and $q_R=0.75$. Compare coverage targets of 90% and 95%. Assume the same unit-variance normal model under both scenarios and a constant $q_R$.

For 90% coverage, $\Phi^{-1}(0.90)=1.2816$, so $c_{90}=0.70+1.2816=1.9816$. The responsive-scenario standardized threshold is $1.9816-1.80=0.1816$, giving

\operatorname{Power}_R(c_{90}) =0.75\{1-\Phi(0.1816)\} \approx 0.75(0.4280) =0.3210.

For 95% coverage, $\Phi^{-1}(0.95)=1.6449$, so $c_{95}=0.70+1.6449=2.3449$. The responsive-scenario standardized threshold is $2.3449-1.80=0.5449$, giving

\operatorname{Power}_R(c_{95}) =0.75\{1-\Phi(0.5449)\} \approx 0.75(0.2929) =0.2197.

Raising the required coverage from 90% to 95% increases the threshold by $2.3449-1.9816=0.3633$ score units and reduces modeled power by about $0.1013$, or 10.13 percentage points. The comparison makes the cost of a more protective requirement visible; whether that cost is acceptable is a design and clinical judgment.

Worked Example: Enforcing Coverage in Two Protected Scenarios

Suppose two protected scenarios must be covered: $B_1$, with $\mu_{B_1}=0.20$ and required coverage 90%, and $B_2$, with $\mu_{B_2}=0.80$ and required coverage 95%. The responsive scenario has $\mu_R=2.10$, and $q_R=0.85$. Assume the stated normal model for each scenario.

The first requirement implies

c\ge 0.20+\Phi^{-1}(0.90)=0.20+1.2816=1.4816.

The second implies

c\ge 0.80+\Phi^{-1}(0.95)=0.80+1.6449=2.4449.

Both requirements hold only when $c\ge\max(1.4816,2.4449)=2.4449$. Since power decreases with $c$, choose $c^*=2.4449$. Coverage is $\Phi(2.4449-0.80)=\Phi(1.6449)=0.9500$ in $B_2$, and $\Phi(2.4449-0.20)=\Phi(2.2449)\approx0.9876$ in $B_1$. The responsive-scenario standardized threshold is $2.4449-2.10=0.3449$, so

\operatorname{Power}_R(c^*) =0.85\{1-\Phi(0.3449)\} \approx 0.85(0.3650) =0.3103.

The stricter binding constraint is the one for $B_2$. Choosing a threshold based only on $B_1$ would fail the 95% requirement for $B_2$.

Using the Tradeoff in Design

The constrained optimum is useful when a minimum protection standard has been set: among thresholds meeting that standard, it retains the greatest modeled power under the assumptions. It does not determine what the minimum should be. That choice must reflect the clinical importance of avoiding exclusion, as well as the consequences of reduced power and delayed or missed enrichment.

There are several reasonable alternatives, each answering a different design question:

Approach	How the threshold is chosen	What to watch
Coverage-constrained power	Maximize responsive-path power subject to specified minimum coverage.	Results depend on the chosen coverage target and scenario model.
Weighted utility	Trade off power and coverage using explicit values assigned to each.	Weights can be difficult to justify and results may be sensitive to them.
Robust coverage	Meet the requirement under the least favorable scenario in a set of plausible models.	A very conservative scenario can substantially reduce power.
Fixed clinical threshold	Use a threshold chosen for clinical or operational reasons rather than optimized probabilities.	Its operating characteristics still need evaluation under plausible scenarios.

A weighted utility could, for example, be written as $U(c)=w_P\operatorname{Power}_R(c)+w_C C_B(c)$, where $w_P$ and $w_C$ are prespecified nonnegative weights reflecting the design priorities. Unlike the constrained approach, this does not guarantee a minimum coverage unless a constraint is added. A robust approach instead requires $C_{B_j}(c)\ge C_{\min,j}$ for every specified scenario $B_j$; under the normal model, the smallest feasible threshold is the maximum of the scenario-specific lower bounds.

Diagnostics and Failure Modes

The calculations are only as useful as the assumptions behind them. Before adopting a threshold, examine whether the planned score is calibrated and whether its distribution under each scenario is plausible. Simulations based on the full interim decision rule can estimate coverage and power directly, including features omitted by a simple normal model. Compare simulated operating characteristics with the formulas, especially near the proposed threshold.

Mis-specified scenario means or variances. If the protected scenario produces larger evidence scores than assumed, its actual coverage may fall below the target. Recalculate or simulate over plausible parameter values.
Confusing a minimum with an optimum. A threshold meeting a coverage target is not automatically clinically optimal. The target encodes a design priority; it should not be chosen merely because it makes the equation convenient.
Ignoring threshold-dependent confirmatory power. If enrollment, multiplicity adjustment, or the confirmatory analysis changes with the threshold, $q_R$ may not be constant. The product $q_RP(S>c\mid R)$ must then be evaluated using the appropriate $q_R(c)$.
Using marginal normal formulas for a more complex rule. If the actual rule combines several scores, uses repeated interim looks, or has additional decision branches, a single-score formula may not represent its operating characteristics. Simulate the specified rule.
Relying on a fragile separation of scenarios. When $\mu_R$ is close to $\mu_B$, the score may have little ability to distinguish responsive from protected scenarios. The formulas still apply, but the attainable power at the chosen coverage can be low.

The threshold also depends on the units and calibration of $S$. If the score is rescaled, its means and threshold must be rescaled together; a numerical cutoff cannot be transferred to a differently standardized score without recalculation. Report the scenario definitions, model assumptions, target coverage, chosen threshold, and resulting power together. This makes clear which tradeoff the design actually accepts.

Key takeaway: Under the specified normal model, increasing $c$ improves protection against excluding the protected scenario and reduces the chance of selecting the responsive population. With a coverage target strictly between 0 and 1 and constant positive $q_R$, the power-maximizing finite threshold is the smallest one that meets the target. The target and model, not the algebra alone, determine whether the resulting design is appropriate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

With $\mu_B=0.50$, $\mu_R=1.50$, $q_R=0.80$, and $C_{\min}=0.90$, find the optimal threshold and its responsive-path power.
Can any finite threshold provide exactly 100% coverage in the normal model? Explain.
If the required coverage increases from 90% to 95%, what happens to the optimal threshold, and why?
Why might the constrained-optimum result fail to identify the power-maximizing threshold if $q_R$ changes with $c$?

Solutions. (1) The threshold is $c^*=0.50+\Phi^{-1}(0.90)=0.50+1.2816=1.7816$. The responsive standardized threshold is $1.7816-1.50=0.2816$, so power is $0.80\{1-\Phi(0.2816)\}\approx0.80(0.3891)=0.3113$, or 31.13%. (2) No. For every finite $c$, $\Phi(c-\mu_B)<1$; coverage approaches 1 only as $c$ tends to infinity. (3) The threshold increases by $\Phi^{-1}(0.95)-\Phi^{-1}(0.90)=1.6449-1.2816=0.3633$, because a higher minimum coverage requires a larger cutoff. (4) If $q_R=q_R(c)$, total power is $q_R(c)\{1-\Phi(c-\mu_R)\}$. Its decrease or increase depends on both factors, so the smallest feasible threshold need not maximize their product; the threshold-dependent power must be optimized directly subject to the coverage constraint.