Gatekeeping Tests Across All-Comer And Enriched Populations

Why Order Confirmatory Claims?

An enrichment trial may be designed to support a treatment claim for all eligible patients, for a biomarker-positive subgroup, or for both. Each claim has its own null hypothesis. If investigators test several claims and report whichever ones are significant, the chance of making at least one false claim can exceed the nominal significance level. A gatekeeping procedure limits that risk by specifying in advance which hypothesis must be rejected before another claim may be tested.

This tutorial focuses on a fixed sequence in which the all-comer claim is tested first and the biomarker-positive claim is tested only if the all-comer test succeeds. This order is appropriate when broad-population efficacy is the primary objective and a subgroup claim is a secondary objective that should not stand alone. If the scientific or regulatory objective is different, a different order may be prespecified; the order determines which claims can be reached.

Here, \(\theta\) is defined so that positive values favor treatment, in the outcome's stated units. The biomarker-positive population must be defined clearly, and the endpoint, analysis, direction of benefit, significance level, and test order should be prespecified. The all-comer population includes the biomarker-positive subgroup; therefore, the two test statistics may be correlated. The fixed-sequence procedure does not require them to be independent.

The procedure also requires valid p-values for the hypotheses being tested. For example, the underlying randomized comparison, endpoint model, standard error, missing-data approach, and any interim adaptation must support the stated test. If a subgroup was selected using accumulating outcome data, an ordinary subgroup p-value may not be valid after selection. Gatekeeping addresses multiplicity among the stated claims; it does not repair invalid inference caused by data-dependent selection.

How the Fixed Sequence Works

Choose a one-sided significance level \(\alpha\), such as 0.025. Test the all-comer hypothesis first. If its p-value is at most \(\alpha\), reject \(H_{0A}\) and proceed to test \(H_{0B}\) at \(\alpha\). If the subgroup p-value is also at most \(\alpha\), reject \(H_{0B}\) and make both claims. If either test fails at its stage, do not make the claim associated with that test or any later test in the sequence.

The subgroup p-value may still be calculated and reported descriptively when the gate is closed, but it does not support a confirmatory subgroup claim under this procedure. That distinction prevents a small downstream p-value from being treated as if it had passed a prespecified testing rule.

Why the Sequence Controls Familywise Error

The familywise error rate is the probability of rejecting at least one true null hypothesis in the family. In the two-claim procedure, the family consists of \(H_{0A}\) and \(H_{0B}\). Strong control at level \(\alpha\) means that this probability is at most \(\alpha\) for every configuration of true and false null hypotheses.

Assume each test has a valid p-value: whenever its null hypothesis is true, the probability that its p-value is at most \(\alpha\) is no greater than \(\alpha\). Consider any configuration of true and false hypotheses. If both nulls are false, there is no type I error to make. Otherwise, find the first true null in the prespecified sequence. A false rejection of any true null can occur only if the procedure reaches and rejects that first true null: if it is not rejected, the sequence stops, so no later hypothesis is tested. Consequently, the event of at least one false rejection is contained in the event that the first true null has a p-value at most \(\alpha\). Thus

where \(H_{0j}\) is the first true null in the sequence. This argument does not use independence between the tests. It relies on valid tests and the rule that no later hypothesis is tested unless every earlier hypothesis has been rejected.

Choosing and Interpreting the Order

The order should reflect which claims the trial is intended to support, not which test appears more likely to succeed after looking at the results. An all-comer-first sequence makes a broad-population claim a prerequisite for the subgroup claim. In a trial whose primary objective is specifically the biomarker-positive population, a subgroup-first sequence may instead be appropriate, with an all-comer claim permitted only after subgroup success. The same fixed-sequence logic applies, but the permitted claims change with the order.

Assumptions, Diagnostics, and Alternatives

Before interpreting a fixed-sequence result, check the conditions that make each component test valid. In a simple randomized trial with a continuous outcome, these may include correct specification of the treatment contrast, a defensible standard error, and an adequate normal approximation. For a different endpoint or analysis model, the relevant assumptions differ. The p-value must correspond to the stated one-sided hypothesis and must account for features such as stratified randomization or repeated interim looks when those features affect the test.

Also verify that the all-comer and biomarker-positive estimands are clearly defined and that the biomarker rule is prespecified. A subgroup analysis defined after examining outcomes, an inconsistent endpoint direction, or a test that ignores the design can undermine the p-value even though the gatekeeping arithmetic is correct. A useful audit is to write down the exact hypotheses, test order, \(\alpha\), and stopping rule before looking at final outcomes, then check that the reported claims follow that rule exactly.

A fixed sequence differs from testing every claim freely at the same nominal level: it restricts which claims can be made and protects the familywise error rate through that restriction. Other multiplicity strategies can permit more flexible claim patterns, but they require their own prespecified error-control rules. A fixed sequence is simple and does not require independent test statistics, but it may withhold a scientifically plausible subgroup claim when an earlier broad-population test fails. It also does not automatically cover additional endpoints, doses, subgroups, or alternative analyses outside the stated family.

• Do not treat a downstream nominal p-value as a gated claim. If an earlier test fails, the later confirmatory claim is unavailable under this sequence.
• Do not change the order after seeing results. Choosing the sequence retrospectively defeats the prespecified decision rule.
• Do not confuse error control with evidence of effect. A failed test does not prove the null is true, and a rejected null does not establish clinical importance by itself.
• Do not assume gatekeeping corrects selection bias. If the subgroup or analysis was selected using outcomes, the p-value itself may need a method that accounts for that selection.