Why Secondary Claims Need Protection
A trial may monitor a primary endpoint and several secondary endpoints at interim analyses. A favorable secondary result can be scientifically interesting, but a confirmatory claim about it must respect both sources of multiplicity: the family of endpoints and the repeated opportunities to reject a null hypothesis over time. A protected secondary claim adds a rule to this design: a secondary endpoint is eligible for formal testing only after a prespecified primary endpoint success.
Protection does not mean that secondary tests are exempt from sequential monitoring. Their boundaries, or their valid sequential p-values, must still account for all planned looks at which a secondary claim could be made. Nor does protection mean that alpha can be restarted when the primary endpoint succeeds. The gate determines when a secondary claim may be made; the secondary testing procedure determines how often that claim can be tested without exceeding its error budget.
Specify the Gate and the Sequential Tests
Let \(H_P\) denote the primary null hypothesis and \(H_S\) a secondary null hypothesis. For a one-sided efficacy test, the design might define primary success as crossing the primary efficacy boundary at any planned look. If the primary test does not cross that boundary, no protected secondary claim is allowed. The gate is about formal claims: it does not by itself determine whether investigators may inspect secondary data for safety, monitoring, or other prespecified purposes.
The primary test is calibrated across its planned looks at a total level \(\alpha_P\). The secondary test is calibrated across its own planned looks at a total level \(\alpha_S\). For example, a conventional group sequential design can choose boundaries so that, under the corresponding null hypothesis, the probability of crossing at least one efficacy boundary is no greater than the assigned level. A conservative alternative is to allocate small per-look levels whose sum is no greater than that level; the union bound then guarantees control even when the test statistics across looks are dependent.
For clarity, consider one primary endpoint and a family of secondary endpoints. Suppose each secondary test is available for a formal claim only after primary success. If several secondary hypotheses are tested, their joint procedure must control the familywise error rate (FWER) for that secondary family at level \(\alpha_S\); for example, a prespecified multiplicity graph can assign local levels that sum to no more than \(\alpha_S\). Each local test must then control its own sequential crossing probability over the planned looks.
Why the Gate Controls Familywise Error
The FWER is the probability of making at least one false rejection among the hypotheses in the family. Under strict primary gating, the primary and secondary tests need not have their levels added mechanically in every configuration. The reason is that a false secondary rejection is possible only on a path where the primary gate has opened. The following argument gives strong control, meaning control for every configuration of true and false hypotheses.
Choose primary and secondary sequential procedures with null crossing probabilities at most \(\alpha_P\) and \(\alpha_S\), respectively, over all planned looks. If there is a secondary family, its procedure must control that family at \(\alpha_S\).
Record the primary success rule and its timing in advance. A secondary boundary crossed before the gate opens does not produce a protected claim.
If the primary null is true, any false rejection in the gated family requires a false primary rejection. If the primary null is false, primary rejection is not an error, and false secondary rejections are bounded by the secondary procedure’s across-look familywise level.
More formally, let \(R_P\) be the event that the primary test rejects, and let \(E\) be the event of at least one false rejection in the complete endpoint family. When \(H_P\) is true, any secondary rejection requires \(R_P\), so \(E\subseteq R_P\). Therefore \(P(E)\leq P(R_P)\leq\alpha_P\). When \(H_P\) is false, a primary rejection is not false; any error must be a false rejection among the secondary hypotheses. The secondary family procedure gives \(P(E)\leq\alpha_S\). Thus, if \(\alpha_P\leq\alpha\) and \(\alpha_S\leq\alpha\), then \(P(E)\leq\alpha\) for either configuration, and the procedure strongly controls FWER at \(\alpha\). This argument does not require independence between endpoint statistics or between looks, provided the individual sequential error bounds are valid under their nulls.
This is a property of strict gating, not a general permission to give every test the full alpha level. If secondary claims can be made without primary success, or if the gate does not logically contain every secondary rejection, the event inclusion used above fails. Then the overall family needs a procedure that controls the combined claims, such as a prespecified graphical allocation. The introductory material on graphical procedures explains how endpoint-level allocations can be represented and updated; here, each allocation must still be paired with valid sequential boundaries.
Worked Sequential Examples
Worked Example: A Protected Claim at a Later Look
Suppose a hypothetical trial has one primary and one secondary one-sided efficacy endpoint, with two planned looks. Set the overall target at \(\alpha=0.05\), and use \(\alpha_P=\alpha_S=0.05\). For a transparent, conservative illustration, assign half of each endpoint’s level to each look. Under a standard normal null statistic, each look uses the threshold \(z_{0.975}=1.960\). This Bonferroni construction controls each endpoint’s chance of crossing at either look by at most \(0.025+0.025=0.05\), regardless of dependence between looks.
At look 1, suppose the primary statistic is \(Z_{P,1}=2.08\), while the secondary statistic is \(Z_{S,1}=1.72\). Since \(2.08>1.960\), the primary endpoint meets its prespecified success criterion and the gate opens. The secondary statistic does not cross its boundary, since \(1.72<1.960\), so no secondary claim is made at this look.
At look 2, suppose \(Z_{S,2}=2.14\). The secondary statistic crosses its planned boundary because \(2.14>1.960\). The secondary claim is now permitted, because the primary gate opened at look 1, and the secondary test has crossed a boundary from its prespecified two-look procedure. The secondary test has not received two separate 0.05 budgets: its per-look allocations sum to 0.05. The numerical threshold is illustrative; a design using a less conservative group sequential boundary would need to calibrate that boundary to the same across-look error bound.
Worked Example: Primary Success at the Final Look
Consider another hypothetical two-look trial with the same one-sided Bonferroni thresholds, \(1.960\) at each look for both endpoints. The primary statistics are \(Z_{P,1}=1.81\) and \(Z_{P,2}=2.06\); the secondary statistics are \(Z_{S,1}=2.22\) and \(Z_{S,2}=2.31\).
At look 1, the primary endpoint does not cross its boundary because \(1.81<1.960\). Although the secondary statistic exceeds its boundary, \(2.22>1.960\), the gate remains closed, so this result cannot support a protected secondary claim. At look 2, the primary statistic crosses its boundary because \(2.06>1.960\). The gate opens at that look. If the secondary result is available and the protocol permits its evaluation at the same look after the primary decision, then \(2.31>1.960\) supports a secondary claim under the prespecified rule. If the protocol instead requires secondary testing only at looks after primary success, no secondary test occurs at look 2; the design must state which rule applies.
The secondary statistic at look 1 cannot be treated as though the gate had already opened. Equally, the design cannot silently change from “subsequent looks only” to “the same look is allowed” after seeing the results. Either rule can be prespecified, but the gate timing and secondary look schedule must be part of the design.
Worked Example: Two Secondary Endpoints With Allocated Levels
Suppose a hypothetical trial has one primary endpoint and two secondary endpoints, \(S_1\) and \(S_2\), each monitored at two planned looks. The primary test has level \(\alpha_P=0.05\). Allocate the secondary-family level \(\alpha_S=0.05\) as 0.03 to \(S_1\) and 0.02 to \(S_2\). For a simple conservative design, split each endpoint’s allocation equally over its two looks. The per-look levels are therefore \(0.015\) for \(S_1\) and \(0.010\) for \(S_2\). The corresponding one-sided standard normal thresholds are approximately \(2.170\) and \(2.326\), respectively.
Assume the primary endpoint crosses its efficacy boundary at look 1, so the gate opens. At that look, suppose \(Z_{S_1,1}=2.20\) and \(Z_{S_2,1}=2.18\). Endpoint \(S_1\) crosses because \(2.20>2.170\). Endpoint \(S_2\) does not, because \(2.18<2.326\). At look 2, suppose \(Z_{S_1,2}=2.05\) and \(Z_{S_2,2}=2.40\). Endpoint \(S_1\) does not newly cross, but its earlier crossing remains a rejection under the rule; \(S_2\) crosses because \(2.40>2.326\).
The total secondary allocation is \(0.03+0.02=0.05\), and the per-look levels sum to 0.03 for \(S_1\) and 0.02 for \(S_2\). By the union bound over endpoints and looks, the chance of at least one false secondary rejection is no more than 0.05 under the complete secondary null family. This calculation is conservative and does not rely on independence. In practice, a prespecified graphical or joint sequential procedure may use the allocated levels more efficiently, but it must retain valid across-look control.
Assumptions, Diagnostics, and Alternatives
The argument depends on a few design conditions. First, each sequential test must be calibrated under its own null hypothesis for the actual planned information times, test direction, and stopping rules. For the numerical illustrations, the null statistics at each look are assumed to have standard normal marginal distributions; that assumption makes each one-look tail probability match the chosen threshold. The union-bound guarantee does not require independence across looks, but it does require valid marginal null distributions. A conventional exact group sequential procedure may instead use a joint model for the statistics, often based on cumulative information and independent increments, and calibrate boundaries from that joint distribution.
Second, the primary gate and its timing must be prespecified. Third, the secondary testing schedule must cover every planned opportunity to make a claim. If the primary succeeds at look \(k\), a design may allow secondary testing at look \(k\), only at later looks, or according to another explicit availability rule. Whatever the rule, the secondary procedure cannot be recalibrated after observing when the gate opened unless the recalibration has been justified as part of the original design.
A useful diagnostic is to list every route to a formal claim: which primary event opens the gate, which secondary looks are eligible, and which boundary applies at each eligible look. Check that no route permits an unplanned test, repeats a spent level, or bypasses the gate. Also confirm that a futility decision or an informal favorable trend does not count as primary success unless the protocol explicitly defines it that way. Futility alone is not an efficacy rejection and does not open an efficacy gate.
| Approach | How it handles the claims | Main consideration |
|---|---|---|
| Protected gatekeeping | Secondary claims require primary success; each sequential procedure is calibrated across its looks. | Efficient when secondary claims are meaningful only after primary success; timing must be explicit. |
| Parallel multiplicity control | Allocates error across primary and secondary claims without requiring primary success first. | Can permit secondary claims despite primary failure, but must control the combined family. |
| Fixed sequence | Tests hypotheses in a prespecified order, passing to the next only after rejection. | Simple and stringent; a failure earlier in the sequence blocks later claims. |
| Graphical allocation | Assigns local levels and reallocates them according to prespecified rejection rules. | Can express more flexible priorities, but every local sequential test still needs valid boundaries. |
A common mistake is to say, “The primary endpoint passed, so the secondary endpoint can now be tested at 0.05 at every remaining look.” This treats each remaining look as a new opportunity with a fresh level. The correction is to use the secondary procedure calibrated over the entire planned testing path, or a formally justified conditional procedure specified in advance. A second mistake is to test a secondary endpoint first, then retrospectively describe its rejection as protected because the primary crossed later. The gate defines when a claim is allowed; it cannot be applied after the fact to erase an earlier testing opportunity.
Protected claims also have limitations. A strict gate can reduce the opportunity for secondary claims when primary success is unlikely, even if a secondary effect is substantial. Conservative per-look allocations may further reduce power. Conversely, a boundary calibrated under a wrong information schedule or an invalid null distribution may not control error as intended. Statistical protection concerns the probability of false claims under the specified model and design; it does not establish clinical importance or repair endpoint-selection bias from unrecorded analyses.
Check Your Understanding
Answer each question using the protected-gate and across-look principles.
- A primary test is monitored at three looks. What must be specified before the trial to determine when a secondary claim becomes eligible?
- A secondary test has two looks, each tested at a one-sided level of 0.03. Is its across-look false-rejection probability necessarily at most 0.05? Explain.
- Under the true primary null, explain why strict secondary gating can control the probability of any false rejection even if the secondary null is also true.
- A primary endpoint succeeds at look 2. The protocol says secondary testing may occur only at later looks, and there are no later looks. Can the secondary result at look 2 support a protected claim?
- Two secondary endpoints receive levels 0.025 and 0.025, each monitored at two looks using equal per-look allocations. What is each endpoint’s per-look level, and what does the union bound say about the secondary-family error probability?
Solutions. (1) Specify the primary success criterion, including its sequential rejection boundary and whether success at a given look opens the gate at that same look or only afterward. Also specify which secondary looks are eligible. (2) No. The union bound gives at most \(0.03+0.03=0.06\), which exceeds 0.05; dependence could make the actual probability smaller, but that is not guaranteed by these levels. (3) Any false secondary rejection requires the primary gate to have opened. When the primary null is true, the event of any false rejection is therefore contained in the event of primary rejection, whose probability is controlled by the primary sequential test. (4) No. The stated rule allows only later looks, and none remain; changing the rule after observing the result would not preserve the prespecified gate. (5) Each endpoint receives \(0.025/2=0.0125\) per look. Summing over both endpoints and both looks gives \(4(0.0125)=0.05\), so the union bound controls the secondary-family false-rejection probability at no more than 0.05.