Confidence Intervals After Population Enrichment

What Enrichment Changes

In a biomarker-enrichment design, an interim analysis may indicate that treatment benefit is concentrated in a subgroup, such as patients whose biomarker is positive. The trial then restricts enrollment to that subgroup. The resulting estimate concerns a clearly defined population, but it is not automatically an estimate for all patients originally eligible for the trial. A second issue is selection: if the subgroup continued because its interim estimate looked favorable, that estimate is unusually likely to overstate its true effect.

The interval must therefore be matched to both the target population and the selection rule. An interval for the biomarker-positive treatment effect does not become an all-comer interval merely because the positive subgroup was part of an all-comer trial. Nor does an ordinary interval necessarily retain its advertised coverage after selecting a subgroup based on interim outcomes.

Definition: Let \(B\) denote a baseline biomarker group, measured before treatment, and let \(Y(1)\) and \(Y(0)\) denote the potential outcomes under treatment and control. For a prespecified subgroup \(B=+\), the subgroup treatment effect on an additive outcome scale is \(\theta_+=E[Y(1)-Y(0)\mid B=+]\). The all-comer effect is \(\theta_{\mathrm{all}}=E[Y(1)-Y(0)]\). An interval has conditional coverage \(1-\alpha\) after selection if, for the stated selection event \(S\), \(P_\theta(\theta\in C\mid S)=1-\alpha\) under the model.

If the biomarker groups are exhaustive and mutually exclusive, and \(q=P(B=+)\), then \(\theta_{\mathrm{all}}=q\theta_+ +(1-q)\theta_-\), where \(\theta_-=E[Y(1)-Y(0)\mid B=-]\). This identity is for an additive effect and a defined target population. It shows why an estimate of \(\theta_+\) alone cannot recover \(\theta_{\mathrm{all}}\): the prevalence and the effect in the biomarker-negative group also matter.

Worked Example: The Enriched Effect Is Not the All-Comer Effect

Suppose a hypothetical trial has a biomarker-positive prevalence of \(q=0.30\). The treatment effects are \(4.0\) points in the positive group and \(-1.0\) point in the negative group. The units are points on the trial's outcome scale.

The positive-group effect is \(\theta_+=4.0\) points. The all-comer effect is

\theta_{\mathrm{all}} =q\theta_+ +(1-q)\theta_- =(0.30)(4.0)+(0.70)(-1.0) =1.20-0.70 =0.50\text{ points}.

Thus the enriched-population effect is \(4.0\) points, whereas the all-comer effect is \(0.50\) point. Reporting an interval for \(\theta_+\) as if it estimated \(\theta_{\mathrm{all}}\) would answer the wrong clinical question. The weighted identity also presumes that the subgroup effects and prevalence refer to the same target population and that the effect scale is additive.

Why Interim Selection Affects an Interval

Consider a simple model for an interim estimate \(X\) of \(\theta_+\): \(X\sim N(\theta_+,\sigma^2)\), with known \(\sigma\). Suppose enrichment occurs when \(X>c\sigma\), where \(c\) is a prespecified threshold. Write \(S=\{X>c\sigma\}\) for the event that the positive subgroup is selected. Before selection, \(X\) is centered at \(\theta_+\). Among trials satisfying \(S\), however, values below the threshold have been removed. In particular, when the true effect is near zero, the selected estimates tend to be positive and too large.

The conditional density makes the adjustment explicit. For \(x>c\sigma\), the density of the observed interim estimate given selection is the ordinary normal density divided by the probability of selection:

f_{\theta_+}(x\mid S) = \frac{\phi((x-\theta_+)/\sigma)} {\sigma\{1-\Phi(c-\theta_+/\sigma)\}}, \qquad x>c\sigma.

Here \(\phi\) and \(\Phi\) are the standard-normal density and cumulative distribution function. The denominator depends on the candidate effect being considered. It cannot be discarded when using the selected interim data for inference.

Result: Under this model, a central \(1-\alpha\) conditional confidence set for \(\theta_+\) is obtained by retaining the values \(\theta\) for which

\frac{\alpha}{2} \le \frac{\Phi((x-\theta)/\sigma)-\Phi(c-\theta/\sigma)} {1-\Phi(c-\theta/\sigma)} \le 1-\frac{\alpha}{2}.

The middle expression is the conditional cumulative probability of observing a value no greater than \(x\), given that \(X>c\sigma\), under candidate value \(\theta\). The interval endpoints are found by solving the two boundary equations, usually numerically. This is test inversion: for each candidate effect, compare the observed estimate with its distribution after applying the same selection rule. The conditional interval targets \(\theta_+\) and has coverage conditional on the stated selection event, under the normal model and known variance assumption.

Worked Example: A Selected Estimate and Its Conditional Distribution

Suppose, hypothetically, that the interim estimate is standardized to \(X\sim N(\theta,1)\), and enrichment occurs if \(X>1.645\). The observed standardized estimate is \(x=2.000\). Consider testing the boundary value \(\theta=0\) against a larger effect, conditional on having selected the subgroup.

Under \(\theta=0\), the conditional upper-tail probability is

P_0(X\ge 2.000\mid X>1.645) = \frac{1-\Phi(2.000)}{1-\Phi(1.645)} = \frac{0.0227501}{0.0499849} \approx 0.4551.

The calculation conditions on selection: the numerator is the chance of reaching at least the observed value, and the denominator is the chance of crossing the enrichment threshold. The conditional upper-tail probability is about \(0.4551\), not \(0.0228\). The ordinary one-sided tail probability treats the estimate as if it had been observed without screening. In this selected-data model, it would give substantially different evidence against the boundary value. A full two-sided conditional interval is obtained by inverting the conditional cumulative distribution above, not by relabeling an ordinary normal interval.

Justification: Truncation Produces Selection Bias

The direction and size of the selection effect can be derived directly. Set \(Z=(X-\theta_+)/\sigma\), so \(Z\sim N(0,1)\), and let \(a=c-\theta_+/\sigma\). Selection is equivalent to \(Z>a\), and \(X=\theta_+ +\sigma Z\). For a standard normal variable, integrating \(z\phi(z)\) over the selected tail gives

E[Z\mid Z>a] = \frac{\int_a^\infty z\phi(z)\,dz}{1-\Phi(a)} = \frac{\phi(a)}{1-\Phi(a)}.

The integral equality follows because \(\phi'(z)=-z\phi(z)\), so \(\int_a^\infty z\phi(z)\,dz=\phi(a)\). Therefore,

E[X\mid X>c\sigma] = \theta_+ +\sigma \frac{\phi(c-\theta_+/\sigma)} {1-\Phi(c-\theta_+/\sigma)}.

The added term is positive. Thus the ordinary interim estimate is upwardly biased among selected trials, even if it is unbiased before selection.

Worked Example: Expected Interim Estimate Under a Null Effect

Continue with the hypothetical standardized model \(X\sim N(\theta,1)\), selection threshold \(c=1.645\), and true effect \(\theta=0\). The conditional mean is

E[X\mid X>1.645] = \frac{\phi(1.645)}{1-\Phi(1.645)} \approx \frac{0.1031}{0.0500} \approx 2.063.

The density is approximately \(0.1031\), and the selected-tail probability is approximately \(0.0500\); the ratio is approximately \(2.063\) standard-error units. Thus, among trials selected by this threshold when the true effect is zero, the mean interim estimate is about \(2.063\) standard errors above zero. This is a statement about the average over selected repetitions, not a correction to subtract mechanically from every observed estimate.

Intervals From Independent Post-Enrichment Data

A direct way to avoid conditioning the interval on a selected interim estimate is to base it on an independent confirmatory stage. Suppose the enrichment decision is made using stage 1, and stage 2 then randomizes only biomarker-positive participants. If stage 2 outcomes are independent of stage 1 outcomes, the stage 2 treatment assignment and analysis are not chosen using those stage 2 outcomes, and the stage 2 sampling model is correctly specified, an ordinary interval based solely on stage 2 is valid conditional on the stage 1 selection event. This approach is often less precise than a valid method that uses both stages, but it is transparent.

Worked Example: A Stage 2 Interval for the Enriched Population

In a hypothetical trial, stage 1 selects the biomarker-positive population. Stage 2 then independently randomizes 50 positive patients to treatment and 50 to control. Assume a normally distributed outcome with known standard deviation \(10\) points in each group, independent observations, no informative missing outcomes, and a fixed analysis plan. The stage 2 mean improvement is \(5.2\) points with treatment and \(2.0\) points with control.

The stage 2 estimate of the positive-population treatment effect is \(5.2-2.0=3.2\) points. Its standard error is

SE=10\sqrt{\frac{1}{50}+\frac{1}{50}} =10\sqrt{0.04} =2.00\text{ points}.

A two-sided 95% normal interval is

3.2\pm 1.96(2.00) = 3.2\pm3.92 = (-0.72,\ 7.12)\text{ points}.

Under the stated model and independent-stage assumptions, this interval has 95% coverage for the treatment effect among biomarker-positive patients in the stage 2 target population, conditional on the stage 1 enrichment decision. It does not estimate the all-comer effect. Its width also reflects that stage 2 has only 100 participants; using independent confirmation protects coverage but may cost precision.

Using Both Stages, Diagnostics, and Alternatives

Stage 1 data need not always be discarded. A conditional likelihood can use the interim and later data while accounting for the event that caused enrichment. In the simple model above, the conditional likelihood contribution for the interim estimate is proportional to its normal density divided by \(P_\theta(S)\). With additional stage 2 observations, their likelihood contribution is included as well. Inverting tests based on that conditional likelihood can yield an interval that uses more information, but the selection event and the joint sampling model must be correctly specified. A prespecified analysis plan should state the biomarker rule, the threshold, the target estimand, and how all stages enter the interval.

Approach	What it uses	Coverage interpretation and trade-off
Naive interval from the selected interim estimate	Selected interim data treated as unselected	Generally fails to account for selection; can be too optimistic or shifted.
Conditional test-inversion interval	Selected interim estimate and the explicit selection rule	Targets conditional coverage for the stated selection event under its model; requires correct selection and sampling models.
Independent stage 2 interval	Only post-enrichment data independent of selection	Can retain conditional coverage simply, but may be less precise and targets the enriched population.
Unconditional interval	The full design, including selection and nonselection paths	Targets coverage averaged over the adaptation process; it need not give the same coverage conditional on enrichment.

Useful diagnostics begin with the protocol and data flow, not just a numerical interval. Verify that the threshold and biomarker definition used in the analysis match those used to select the population; confirm that stage 2 was independent of the interim outcomes as claimed; and check that the reported effect is labeled with its population and units. For a conditional model, inspect whether the assumed outcome distribution and variance are plausible and whether the selection event has been represented exactly. A threshold that was changed after viewing the data, additional unrecorded selection criteria, or a variance estimated as though selection had not occurred can invalidate the stated conditional guarantee.

The distinction between conditional and unconditional coverage is consequential. A procedure with 95% coverage averaged over every possible adaptation path may cover less than 95% among trials that actually enrich. Conversely, a method designed to give conditional coverage after enrichment answers a conditional question and may behave differently before selection. The intended guarantee should be stated rather than described simply as “a 95% interval.”

Common Mistakes and Limits

Calling the enriched effect the overall effect. The positive-group estimand is conditional on biomarker status. An all-comer effect needs information about both subgroup effects and their prevalence, or other justified assumptions.
Using the ordinary interim interval after selecting on its estimate. Selection truncates the estimate's distribution. Use a method that accounts for the selection event or independent post-selection data.
Subtracting a fixed selection-bias amount. The conditional bias depends on the true effect, variance, and threshold. The truncated-normal expectation is a model result, not a universal correction for each observed trial.
Ignoring the selection rule in an otherwise sophisticated calculation. Conditional inference is only as accurate as the recorded decision rule and its assumed probability model.
Claiming a conditional guarantee from an unconditional analysis. Coverage averaged over adaptation paths does not, by itself, establish coverage among enriched trials.

These methods do not repair an ill-defined biomarker subgroup, a changing outcome definition, or informative loss to follow-up. Nor can positive-subgroup data alone identify an all-comer effect when the negative subgroup's effect is unknown. If the target is the overall population, the analysis needs suitable information on that population and must account for the design features that generated it.

Check Your Understanding

Answer each question, then compare with the brief solutions. The effects below use an additive outcome scale.

A biomarker-positive group makes up 40% of the target population. Its treatment effect is 3 points, and the negative-group effect is 1 point. What is the all-comer effect?
Why can the mean interim estimate among trials selected for a high positive result exceed the true effect?
Under \(X\sim N(\theta,1)\) and selection \(X>1.645\), an observed \(X=2.000\) is evaluated at \(\theta=0\). What conditional upper-tail probability should be used?
Stage 2 begins after enrichment and uses new, independent randomized observations. What assumptions allow a stage 2-only interval to have coverage conditional on the interim selection?
Can data collected only in the enriched positive group identify the all-comer treatment effect without information or assumptions about the negative group? Explain.

Solutions. (1) \(0.40(3)+0.60(1)=1.8\) points. (2) Selection removes estimates below the threshold; in the normal model, \(E[X\mid X>c\sigma]=\theta+\sigma\phi(c-\theta/\sigma)/\{1-\Phi(c-\theta/\sigma)\}\), which exceeds \(\theta\). (3) The conditional probability is \(\{1-\Phi(2.000)\}/\{1-\Phi(1.645)\}=0.0227501/0.0499849\approx0.4551\). (4) The stage 2 data must be independent of the interim selection data; randomization, sampling, outcome model, and analysis must be correctly specified, with no stage 2 outcome-driven selection or unaccounted missingness. (5) No. The all-comer effect also depends on the negative-group effect and subgroup prevalence; positive-group observations alone do not supply the negative-group effect.