Why Pocock Uses A Flat Z Boundary

Why a Flat Cutoff Is Relatively Permissive Early

A Pocock design uses the same critical Z value at every planned analysis. That simple rule has an important consequence: its early cutoff is lower than the very high early cutoff of an O’Brien-Fleming design calibrated to the same overall type I error. The reason is not that Pocock assigns a separate, equal share of type I error to each look. Rather, the cutoff is calibrated against the cumulative probability of crossing at any look. A flat threshold allows some of that cumulative probability to arise early; an O’Brien-Fleming design keeps early crossing much rarer and allows more of the error probability to accrue near the final analysis.

The comparison concerns efficacy stopping under the null hypothesis. It assumes a planned set of analyses, a valid sequence of Z statistics, and a one-sided upper rejection boundary. The threshold is “less stringent” when its required Z value is lower, so that an observed statistic can cross it more readily.

Model and Cumulative Crossing Probability

Definition: Let $Z_1,\ldots,Z_K$ be the standardized test statistics at $K$ planned analyses, and let $t_1<\cdots<t_K=1$ be their information fractions. Under the global null and the canonical group sequential model, the statistics are jointly standard normal, with $$ \operatorname{Corr}(Z_i,Z_j)=\sqrt{\frac{t_i}{t_j}}, \qquad i<j. $$ For one-sided upper-boundary testing, the cumulative crossing probability for cutoffs $b_1,\ldots,b_K$ is $$ P_{H_0}\left(\bigcup_{k=1}^{K}\{Z_k\ge b_k\}\right). $$ A Pocock design uses a common cutoff $c$, chosen so this probability equals the target familywise type I error $\alpha$.

This canonical model follows when information increases as planned, the null increments are independent, and the standardized statistics are formed from those increments in the usual way. The correlation is positive because later statistics include information already represented in earlier statistics. In particular, the looks are not independent. If the analysis uses a different model, the joint null distribution and the calibration must match that model.

For each look $k$, define its first-crossing probability as the probability of crossing at that look without having crossed earlier:

q_k=P_{H_0}(Z_1<c,\ldots,Z_{k-1}<c,\ Z_k\ge c).

These first-crossing events are disjoint, and their union is the event of crossing at least once. Therefore,

P_{H_0}\left(\max_{1\le k\le K} Z_k\ge c\right) =\sum_{k=1}^{K}q_k.

This identity explains why equal cutoffs do not imply equal error probabilities at each look. The $q_k$ depend on the joint distribution and on whether earlier boundaries were crossed. The marginal probability $P_{H_0}(Z_k\ge c)$ is not generally the probability of a first crossing at look $k$.

Two-Look Derivation

With two looks, information fractions $t_1$ and $t_2=1$, the canonical correlation is $\rho=\sqrt{t_1}$. If the common Pocock cutoff is $c$, the probability of no crossing is the bivariate normal probability that both statistics lie below $c$. Thus the overall crossing probability is

P_{H_0}(Z_1\ge c\ \text{or}\ Z_2\ge c) =1-\Phi_2(c,c;\rho), \qquad \rho=\sqrt{t_1},

where $\Phi_2(c,c;\rho)=P(Z_1<c,Z_2<c)$ for standard normal variables with correlation $\rho$. Calibration chooses $c$ to make this expression equal to $\alpha$. Since a first-look crossing is part of the overall crossing event, $P(Z_1\ge c)$ is one contribution; additional probability comes from crossing at the second look after not crossing at the first.

The comparison with O’Brien-Fleming follows from how the two designs manage this cumulative probability. Pocock keeps the same cutoff, so the early threshold is not inflated to suppress early crossing. O’Brien-Fleming uses a much higher early threshold and a lower threshold near the final analysis. It therefore spends very little of the cumulative type I error early. This is a comparison of boundary shapes, not a claim that either design has a different target overall $\alpha$.

Worked Example: A Two-Look Pocock Cutoff

Consider a hypothetical one-sided design with two looks at $t_1=0.50$ and $t_2=1.00$, targeting $\alpha=0.025$. Suppose numerical evaluation of the canonical bivariate normal probability gives a calibrated common cutoff $c\approx2.178$. The correlation is $$ \rho=\sqrt{\frac{0.50}{1.00}}=0.7071. $$

The calibration is the equation $$ 1-\Phi_2(2.178,2.178;0.7071)\approx0.025. $$ At either look, the marginal upper-tail probability is approximately $P(Z\ge2.178)=0.0147$. At the first look this is also the first-crossing probability, so $q_1\approx0.0147$. The probability of first crossing at the second look is the remaining part of the familywise error: $$ q_2\approx0.0250-0.0147=0.0103. $$

The design therefore has a flat cutoff but not equal first-crossing probabilities: about 0.0147 of the null crossing probability occurs at the first look and about 0.0103 at the second. The figures are approximate, reflecting numerical calibration and rounding. In context, the design has a 2.5% probability of at least one upper-boundary crossing across the two looks when the global null is true.

Worked Example: Comparing Early Stringency

Continue with the same hypothetical two-look design and its Pocock cutoff of approximately 2.178. A conventional O’Brien-Fleming design with the same overall error target has a substantially higher early boundary; for illustration, take an early cutoff of 2.80. The marginal null tail probabilities at the first look are $$ P(Z_1\ge2.178)\approx0.0147, \qquad P(Z_1\ge2.80)\approx0.00256. $$

Thus the Pocock early cutoff permits a first-look null crossing probability about $$ \frac{0.0147}{0.00256}\approx5.74 $$ times as large as the illustrative O’Brien-Fleming early cutoff. This is a comparison of early stringency: the lower Pocock threshold is easier to cross. It does not mean that the Pocock design has a larger overall type I error; both boundaries are calibrated to the same familywise target across all planned looks. The O’Brien-Fleming design compensates for its very small early crossing probability by using a less demanding boundary later.

The numerical O’Brien-Fleming cutoff here is illustrative of a two-look boundary, not a universal value. Its exact value depends on the design’s specified information times and calibration. The point is the shape comparison: its early threshold is higher than the flat Pocock threshold at the same overall error target.

Why Dependence Matters

A tempting shortcut is to calculate the marginal tail probability at each look and treat the looks as independent. The canonical correlation shows why this is wrong. When the statistics are positively correlated, crossing at one look provides information about the chance of crossing at another. The familywise probability must be computed from the joint distribution, or by an equivalent valid sequential calculation.

Worked Example: The Independent-Looks Shortcut

Using the two-look Pocock example, each marginal tail probability is about 0.0147. If the looks were incorrectly treated as independent, the estimated probability of at least one crossing would be $$ 1-(1-0.0147)^2 =1-(0.9853)^2 \approx0.0292. $$

That exceeds the calibrated familywise target of 0.025. Under the canonical model, the correlation is approximately 0.7071, and the joint calculation instead gives an overall crossing probability of approximately 0.025. The discrepancy arises because the independent calculation uses the wrong joint distribution. The marginal tail probability is still useful for describing the chance of crossing at a particular look, but it cannot by itself determine the familywise error across dependent looks.

Interpretation, Diagnostics, and Limitations

The cumulative-probability view gives a practical interpretation of the Pocock rule: it uses a constant Z threshold, and calibration determines how the total null crossing probability is distributed over the planned analyses. It does not promise equal first-crossing probabilities, equal increments in error spending, or equal conditional crossing probabilities. Those quantities depend on the look schedule and the joint null distribution.

Before relying on a flat boundary, check that the design’s planned information fractions and number of looks match the calibration. Also verify that the test statistic and null covariance structure are appropriate. If information times change, looks are added, or the covariance differs from the canonical model, the original cutoff need not control the intended familywise error.

The main alternative in this comparison is the O’Brien-Fleming approach: it protects the early looks with higher thresholds and is less demanding near the final look. Pocock’s constant boundary instead has a lower early threshold, while maintaining the target overall error through joint calibration. Neither shape is uniformly best. A flat boundary can make early stopping for efficacy more attainable, but its use still requires a prespecified design and valid monitoring. This discussion concerns efficacy boundaries; it does not establish stopping rules for futility or address sample-size re-estimation.

Common mistake: “The same cutoff at every look means the same amount of type I error is spent at every look.” The correction is that a constant cutoff gives equal thresholds, not equal first-crossing probabilities. The overall error is the probability of crossing at least once, calculated using the dependent joint distribution of the Z statistics.

Specify the planned looks and null model.
Record the information fractions and verify the assumed joint distribution of the sequential Z statistics.

Set the common threshold.
For a Pocock design, find the cutoff $c$ for which the null probability of crossing at least one planned upper boundary equals $\alpha$.

Interpret the early boundary.
Compare the flat cutoff with a design that deliberately sets a higher early threshold. The Pocock threshold is less stringent early, even though the calibrated overall error target is the same.

Check Your Understanding

Answer each question using cumulative crossing probability, not isolated marginal tail probabilities. Brief solutions follow the questions.

In a two-look design with a common cutoff $c$, express the overall null crossing probability using the bivariate normal CDF and the correlation $\rho$.
Why does a constant cutoff not imply equal first-crossing probabilities at each look?
A design has target familywise $\alpha=0.025$, with approximate first-crossing probabilities $q_1=0.0147$ and $q_2=0.0103$. What is the overall crossing probability?
Why is it incorrect to use $1-(1-P(Z\ge c))^K$ for the familywise error in a canonical group sequential design?
In one sentence, explain what “less stringent early” means when comparing Pocock with O’Brien-Fleming.

Solutions. (1) The probability is $1-\Phi_2(c,c;\rho)$, where $\rho=\sqrt{t_1/t_2}$. (2) First-crossing probabilities depend on the joint distribution and the requirement that no earlier boundary was crossed; equal thresholds do not make those probabilities equal. (3) The disjoint first-crossing probabilities add: $0.0147+0.0103=0.0250$. (4) That expression assumes independent looks, whereas canonical sequential Z statistics are positively correlated, with $\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}$. (5) Pocock’s early Z cutoff is lower and therefore easier to cross than the higher early O’Brien-Fleming cutoff, while both designs can be calibrated to the same overall type I error.