Why Enrichment Diagnostics Matter
An enriched trial recruits or analyzes patients according to a biomarker-defined subgroup, often to improve the chance of detecting benefit where the treatment effect is expected to be larger. A design can nevertheless fail for different reasons: too few eligible patients may be available, the treatment effect may not differ enough across biomarker groups, the assay may classify patients inaccurately, a data-dependent rule may select different subgroups under small perturbations, or recruitment may be slower than projected. These are distinct problems. A corrective analysis should match the suspected failure rather than treating every disappointing result as a power problem.
The earlier material on treatment-effect heterogeneity, simulation operating characteristics, prevalence drift, external cutpoint validation, and imperfect biomarker classification provides the relevant design concepts. Here the emphasis is on tracing a weakness in a design or its results to a plausible cause and identifying a targeted diagnostic. The calculations below use hypothetical data.
Diagnose the Failure Before Changing the Design
The central question is not simply whether a planned sample size was reached. It is whether the design's operating assumptions remain plausible and which assumption is driving poor performance. For example, fewer biomarker-positive participants than expected can reflect a low underlying prevalence, a changing prevalence over calendar time, or both. A small estimated subgroup contrast can reflect genuinely weak heterogeneity, imprecise estimation, or assay error. Each explanation calls for a different check.
For a binary endpoint or continuous endpoint, the particular analysis model must match the outcome and randomization scheme. The diagnostic principles below do not replace that model. For clarity, the treatment-effect calculations in the second example assume a continuous endpoint, independent observations, equal known standard deviation in groups, and randomized allocation within each biomarker subgroup.
Prevalence and Recruitment Feasibility
If each screened patient independently has biomarker-positive status with probability \(\pi\), then the number of positive patients among \(N\) screened follows \(X\sim\operatorname{Binomial}(N,\pi)\), with mean \(N\pi\) and variance \(N\pi(1-\pi)\). This model assumes a stable prevalence in the screened population, independent classifications, and no additional eligibility losses. If these assumptions are implausible, the simple binomial calculation is only a starting point.
A design that requires \(m\) positive patients should be assessed by the probability of obtaining at least \(m\), not just by whether its expected count \(N\pi\) equals \(m\). In practice, screening losses, consent, assay failure, and site-specific recruitment rates also reduce the number randomized. When prevalence may change over time, evaluate recruitment by calendar period or site as well as in aggregate.
Worked Example: Low Prevalence Makes a Recruitment Target Fragile
Suppose a hypothetical trial needs 240 biomarker-positive participants and expects to screen 3,600 eligible patients. Assume initially that classifications are correct, prevalence is constant, and patient biomarker statuses are independent. At prevalence \(\pi=0.08\), the expected number positive is \(3{,}600(0.08)=288\). The standard deviation is \(\sqrt{3{,}600(0.08)(0.92)}=16.28\).
Using a normal approximation with continuity correction, the probability of reaching at least 240 is approximately \(\displaystyle P(X\ge240)\approx P\left(Z\ge\frac{239.5-288}{16.28}\right)=P(Z\ge-2.979)\approx0.9986.\) Under these assumptions, the screening target has a substantial buffer.
Now suppose prevalence is 0.05. The expected positive count is \(3{,}600(0.05)=180\), with standard deviation \(\sqrt{3{,}600(0.05)(0.95)}=13.08\). The same approximation gives \(\displaystyle P(X\ge240)\approx P\left(Z\ge\frac{239.5-180}{13.08}\right)=P(Z\ge4.55)\approx0.000003.\) The trial is exceedingly unlikely to reach its target in this scenario. The corrective analysis is to revise the prevalence assumption using current screening data and recalculate screening requirements, recruitment time, or the target sample size. The calculation does not account for consent or other losses, which would make feasibility worse.
Weak Heterogeneity Is Not the Same as No Effect
An enriched strategy is most compelling when the treatment effect differs meaningfully between biomarker groups, not merely when one subgroup estimate is statistically significant and another is not. The relevant quantity is the interaction contrast \(\Delta=\tau_+-\tau_-\), with its uncertainty. A wide interval can indicate insufficient information to distinguish weak heterogeneity from a clinically important difference.
The all-comer average treatment effect is \(\pi\tau_+ +(1-\pi)\tau_-\), assuming the subgroup effects apply to the target population and the biomarker groups exhaust that population. A large effect in the positive group can coexist with little overall benefit when prevalence is low. Conversely, if effects are similar across groups, restricting enrollment may lose eligible participants without concentrating benefit. Interpretation also depends on the clinical consequences of excluding the biomarker-negative group.
Worked Example: A Noisy Estimate of the Interaction
In a hypothetical continuous-outcome trial, each treatment arm has 100 patients in each true biomarker subgroup. The endpoint standard deviation is assumed to be 10 units in all four groups, observations are independent, and allocation is randomized within subgroup. The estimated treatment effect is 5 units in the positive subgroup and 3 units in the negative subgroup, so the estimated interaction is \(5-3=2\) units.
The standard error of each subgroup treatment-effect estimate is \(\displaystyle SE(\hat{\tau}_+)=SE(\hat{\tau}_-)=\sqrt{\frac{10^2}{100}+\frac{10^2}{100}}=\sqrt{2}=1.414\text{ units}.\) Because the subgroup estimates use disjoint patients, the interaction standard error is \(\displaystyle SE(\hat{\Delta})=\sqrt{1.414^2+1.414^2}=2.000\text{ units}.\)
The approximate 95% confidence interval is \(2\pm1.96(2)=[-1.92,5.92]\) units, and the two-sided normal-test statistic is \(z=2/2=1.00\), giving \(p\approx0.317\). The estimate is compatible with no interaction and also with a positive interaction as large as about 5.92 units. It does not establish that effects are equal. A useful correction is to report the interaction estimate and interval, examine whether the design had adequate precision for a clinically relevant interaction, and avoid making an enrichment claim from separate within-subgroup significance tests.
Assay Misclassification Changes Who Is Enriched
For a binary assay, sensitivity is \(Se=P(A=1\mid B=1)\) and specificity is \(Sp=P(A=0\mid B=0)\), where \(A=1\) denotes an observed positive result. Given prevalence \(\pi\), the positive predictive value is \(\displaystyle P(B=1\mid A=1)=\frac{Se\pi}{Se\pi+(1-Sp)(1-\pi)}.\) Thus even a reasonably accurate assay can yield a notable fraction of false positives when prevalence is low. If treatment effects differ by true status, the observed positive-group effect is a mixture of the two true-group effects.
The following mixture calculation assumes misclassification is independent of randomized treatment and outcome conditional on true biomarker status, and that the assay characteristics apply to the trial population. If these conditions fail, the simple mixture is not adequate. Validation data or sensitivity analyses across plausible sensitivity and specificity values are then especially important.
Worked Example: Quantifying Dilution in the Assay-Positive Group
Consider a hypothetical population with prevalence \(\pi=0.20\), assay sensitivity \(Se=0.85\), and specificity \(Sp=0.90\). Among 1,000 patients, expected true positives number \(1{,}000(0.20)=200\), of whom \(200(0.85)=170\) test positive. Of 800 true negatives, \(800(0.10)=80\) test positive falsely. Therefore the positive predictive value is \(170/(170+80)=0.680\).
Suppose, hypothetically, the treatment effects are 6 units for true-positive patients and 1 unit for true-negative patients. Under the stated nondifferential-error assumptions, the effect among assay-positive patients is \(\displaystyle 0.680(6)+0.320(1)=4.400\text{ units}.\) Among assay-negative patients, expected true positives are \(200(0.15)=30\), and true negatives are \(800(0.90)=720\). The true-positive fraction is therefore \(30/(30+720)=0.040\), giving an observed assay-negative treatment effect of \(0.040(6)+0.960(1)=1.200\) units. The observed contrast is \(4.400-1.200=3.200\) units, rather than the true contrast of \(6-1=5\) units.
This calculation illustrates dilution, not a universal correction factor. A corrective analysis should estimate assay performance using validation data where possible, propagate uncertainty in sensitivity and specificity, and show how treatment-effect conclusions change across plausible assay-error values. Correcting a point estimate while ignoring uncertainty in the validation estimates can make the result appear more precise than it is.
Check Selection Stability and Recruitment Assumptions
An enrichment rule may depend on a biomarker threshold selected from data or on an estimated subgroup effect. Such a rule is unstable when modest changes in the data produce substantially different thresholds, eligible populations, or treatment recommendations. Selection-induced optimism is a related concern: the subgroup chosen for having the strongest estimated effect tends to have an overestimated effect. A reanalysis that merely reports the selected subgroup's estimate does not resolve that problem.
A practical diagnostic is to repeat the complete selection procedure across simulated datasets or resampled datasets and record how often each rule is selected, the resulting sample size, and the performance of the selected rule on independent or appropriately held-out data. For a design evaluation, simulations should include the selection step itself. Bootstrap resampling can reveal sensitivity to the observed sample, but it cannot by itself establish external validity or compensate for an unrepresentative source sample.
Worked Example: Detecting an Unstable Threshold Rule
A hypothetical design compares three prespecified biomarker thresholds. In 1,000 simulations under a plausible treatment-effect scenario, threshold A is selected 510 times, B is selected 330 times, and C is selected 160 times. These frequencies sum to \(510+330+160=1{,}000\), so the selection proportions are 0.510, 0.330, and 0.160.
If the intended rule is expected to identify a consistent population, selection of A in only about half the replications is a warning: the choice is sensitive to sampling variation under this scenario. The next checks are not to declare A correct because it was selected most often, but to examine selection frequencies across the full scenario grid, the selected effect's bias, power and coverage, and the consequences of selecting each alternative. If the rule is data-dependent, its confirmatory error properties must be evaluated with that selection incorporated. Independent validation or a simpler prespecified rule may be preferable when the threshold remains unstable.
Accrual assumptions also need diagnostics beyond a single average recruitment rate. Compare projected and observed screening, assay-success, eligibility, consent, and randomization rates by site and calendar time. A rate estimated from a short start-up period or a small number of high-performing sites may not represent later recruitment. Corrective analyses can use conservative rates, site-level variation, and plausible prevalence drift in predictive simulations; they should also quantify the implications for trial duration and the chance of meeting the target.
A Targeted Diagnostic Workflow
The following sequence keeps the diagnosis tied to the design failure. It is not a substitute for clinical judgment: a statistically feasible design may still be clinically unattractive, and a design with uncertain performance may warrant a different population or objective.
Compare planned and observed prevalence, subgroup counts, assay success, recruitment rate, and effect estimates. Separate a recruitment failure from a weak or imprecise treatment-effect contrast.
Assess stability of prevalence, site and calendar-time variation, assay performance, subgroup sample sizes, and whether the analysis accounts for the selection rule. State which assumptions are supported by data and which remain speculative.
Use recruitment-probability calculations for feasibility, interaction intervals for heterogeneity, validation-informed sensitivity analyses for assay error, and full-procedure simulations for rule stability and operating characteristics.
Compare feasible alternatives such as broader recruitment, revised screening targets, a simpler prespecified rule, or additional validation. Report uncertainty and the clinical consequences of each alternative.
Common Mistakes and Limitations
- Wrong: “The positive subgroup is significant and the negative subgroup is not, so the treatment effect differs.” Correction: Compare the subgroup effects directly using an interaction estimate and its uncertainty; different significance labels do not establish a difference.
- Wrong: “Expected positives equal the target, so recruitment is feasible.” Correction: Calculate the probability of reaching the target and include screening, consent, assay failure, and recruitment-rate uncertainty.
- Wrong: “The assay is 90% specific, so nearly all assay-positive patients are truly positive.” Correction: Positive predictive value also depends on prevalence; calculate it using sensitivity, specificity, and the target population's prevalence.
- Wrong: “One threshold was selected most often, so it is stable.” Correction: Examine the full selection distribution, scenario dependence, and performance after selection. A plurality can still reflect substantial instability.
- Limitation: Diagnostic analyses depend on the quality and representativeness of prevalence, validation, and recruitment data. Sensitivity analyses reveal dependence on assumptions but do not identify which assumption is true without relevant evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A trial expects 30% biomarker prevalence but observes 18% during recruitment. Name two diagnostics needed before concluding that the design is infeasible.
- Why is “significant in the positive subgroup, not significant in the negative subgroup” insufficient evidence of treatment-effect heterogeneity?
- With prevalence 0.10, sensitivity 0.90, and specificity 0.90, calculate the positive predictive value.
- A threshold is selected in 95% of simulated replications under one scenario but only 40% under another. What does this say about stability?
- Give one limitation of using a binomial model for the number of biomarker-positive patients screened.
Solutions. (1) Check whether the screened population, assay, and calendar-time distribution support the original prevalence assumption; then recalculate the probability of reaching the target using updated prevalence and other screening losses. A lower estimate alone does not establish infeasibility without the target and recruitment horizon. (2) The two within-group tests address whether each subgroup effect differs from zero, not whether the subgroup effects differ from each other. Estimate the interaction and its uncertainty. (3) \(P(B=1\mid A=1)=0.90(0.10)/[0.90(0.10)+0.10(0.90)]=0.09/0.18=0.500\), so half of assay-positive patients are expected to be truly positive under these assumptions. (4) Stability is scenario-dependent: the rule is frequently selected in one scenario but not the other. Inspect performance and selection uncertainty across scenarios rather than calling the rule universally stable. (5) The binomial model assumes independent patients and constant prevalence. Site clustering or prevalence drift can violate those assumptions and change the probability of reaching the recruitment target.