Why Read a Boundary on the P-Value Scale?
A group sequential design is often specified using standardized-statistic cutoffs: at look \(j\), the trial may stop for efficacy if its statistic exceeds \(c_j\). Investigators and readers, however, frequently encounter results reported as p-values. Translating \(c_j\) to the p-value scale makes the decision rule easier to compare with familiar significance thresholds, while retaining the essential fact that the threshold can depend on the look.
This conversion is a change of scale, not a new calibration of the design. It builds on the earlier material on nominal p-values at interim looks: the nominal p-value is calculated from the current look’s statistic as if that look were considered by itself. It does not, by itself, account for the repeated opportunities to stop across the full trial.
The definition assumes that the design uses a standard normal statistic and a positive, upper-tail efficacy boundary. At each planned look, \(Z_j\) should be approximately \(N(0,1)\) under \(H_0\), and the reported p-value must use the same statistic, direction, and null reference distribution as the boundary rule. The information accumulated at different looks is typically nested, so the statistics across looks are dependent. That dependence matters for the design’s overall type I error, but not for converting a particular cutoff \(c_j\) to its one-look normal tail area.
For a continuous standard normal distribution, the upper-tail probability decreases as the cutoff increases. Therefore a larger \(c_j\) corresponds to a smaller nominal p-value threshold and a more stringent efficacy criterion at that look. Conversely, a smaller cutoff allows efficacy to be declared at a less extreme statistic.
The Conversion and Its Decision Equivalence
The conversion follows directly from the definition of a one-sided p-value. Under the null reference distribution, values at least as large as the observed \(z_j\) provide evidence in the upper-tail direction. The probability of that tail is \(1-\Phi(z_j)\). Applying the same calculation to the boundary itself gives the threshold.
Because \(\Phi\) is strictly increasing, the two decision rules are equivalent:
Thus, the p-value threshold at look \(j\) is not an approximation to the cutoff’s meaning; for this test and reference distribution, it expresses the same boundary on a different scale. Tables of standard normal tail probabilities or statistical software can be used to evaluate \(1-\Phi(c_j)\). If the cutoff is rounded, the resulting threshold is rounded as well.
Worked Example: Converting a Pocock Cutoff
Suppose a hypothetical two-look, one-sided design supplies a Pocock efficacy cutoff of \(c_j=2.178\) at each look. Convert this cutoff to a nominal p-value threshold.
The model for the conversion is a standard normal upper-tail reference distribution, and the alternative is in the positive direction. The supplied cutoff is therefore used with a one-sided tail:
At either look, an observed \(Z_j\) of at least 2.178 has a one-sided nominal p-value no greater than about 0.0147 and crosses this efficacy boundary. The same threshold applies at both looks because the supplied Pocock cutoff is the same at both looks. This is a look-specific nominal threshold; it is not the overall type I error rate for the two-look design.
How Pocock and O’Brien-Fleming Look on the P-Value Scale
The two boundary families are distinguished by how their standardized cutoffs vary over planned looks. Pocock designs typically use cutoffs that are relatively similar across looks. Since the normal upper-tail function maps similar cutoffs to similar tail areas, their nominal p-value thresholds are also relatively similar. A Pocock design therefore does not require an extremely small nominal p-value at the earliest look in the way an O’Brien-Fleming design does.
O’Brien-Fleming designs use a high standardized cutoff early and a lower cutoff near the final analysis. The corresponding nominal p-value threshold is very small early and increases toward the final look. The scale change can make this pattern especially clear: a boundary that looks like a modest change in \(Z\) can correspond to a substantial change in tail probability.
The exact cutoffs depend on the design’s planned looks, information fractions, sidedness, and calibration. The table below uses rounded illustrative cutoffs for a hypothetical two-look, one-sided design. They are given as supplied design values for demonstrating the conversion; they are not universal constants for every Pocock or O’Brien-Fleming design.
| Boundary family and look | Illustrative cutoff \(c_j\) | One-sided nominal threshold \(1-\Phi(c_j)\) | Pattern |
|---|---|---|---|
| Pocock, first look | 2.178 | 0.0147 | Similar threshold at each look |
| Pocock, final look | 2.178 | 0.0147 | Similar threshold at each look |
| O’Brien-Fleming, first look | 2.797 | 0.00258 | Very stringent early threshold |
| O’Brien-Fleming, final look | 1.977 | 0.0240 | Threshold increases toward the final look |
These entries are nominal thresholds at individual looks. They do not imply that the chance of a false positive at a particular look, conditional on reaching it, equals the listed number. Nor should the thresholds be summed to get the overall probability of any false efficacy declaration. The trial’s repeated decisions and the dependence among its test statistics affect that overall probability.
Worked Example: Comparing O’Brien-Fleming Thresholds Across Looks
In a hypothetical two-look O’Brien-Fleming design, suppose the supplied cutoffs are \(c_1=2.797\) at the interim look and \(c_2=1.977\) at the final look. Find both nominal thresholds and describe their change.
For the interim look, use the one-sided upper tail:
For the final look:
The final nominal threshold is about \(0.0240/0.00258\approx 9.30\) times the interim threshold. That ratio describes the two tail areas; it is not a ratio of the design’s overall false-positive probabilities. The small interim threshold reflects the high interim cutoff, while the less extreme final cutoff corresponds to a larger nominal p-value threshold.
Worked Boundary Decisions
To use the p-value scale correctly, compare the observed p-value with the threshold from the same look and design. The observed p-value and the threshold must also use the same tail convention. For an upper-tail efficacy test, both use \(1-\Phi(\cdot)\).
Worked Example: The Same Interim Result Under Two Designs
Suppose a hypothetical interim analysis gives \(z_1=2.35\). Compare the result with the illustrative first-look cutoffs above: Pocock \(c_1=2.178\) and O’Brien-Fleming \(c_1=2.797\). Assume the statistic is standard normal under \(H_0\), the alternative is upper-tailed, and these cutoffs are the applicable efficacy boundaries.
First calculate the observed nominal p-value:
For the Pocock design, \(2.35\ge2.178\), and \(0.00939\le0.0147\). The statistic crosses that design’s efficacy boundary. For the O’Brien-Fleming design, \(2.35<2.797\), and \(0.00939>0.00258\). It does not cross that design’s interim efficacy boundary. The same observed statistic can therefore lead to different boundary decisions because the planned designs impose different interim cutoffs.
This comparison concerns efficacy-boundary crossing only. It does not say whether a trial stops for another reason, and it does not compare the designs’ overall operating characteristics.
Assumptions, Diagnostics, and Common Pitfalls
Before interpreting a converted threshold, check that the calculation matches the design and analysis:
- Reference distribution. The formula \(1-\Phi(c_j)\) applies to a standard normal statistic. If the specified boundary uses another reference distribution or a different statistic, use that distribution’s appropriate tail probability instead.
- Direction and sidedness. The formula is for a one-sided upper-tail efficacy test. A lower-tail test uses the lower tail. A two-sided test requires its stated two-sided convention; for a symmetric normal test with a boundary on \(|Z_j|\), the corresponding tail probability is \(2[1-\Phi(c_j)]\).
- Look and design. Use the cutoff for the actual planned look and the design being evaluated. Do not substitute the final cutoff for an interim cutoff.
- Statistic definition. Confirm that the reported statistic uses the same standardization and direction as the boundary table. A p-value from a different model or test need not reproduce the boundary decision.
- Rounding. If a statistic or cutoff is reported only to a few decimal places and lies very close to the boundary, rounding may obscure whether it crosses. Use unrounded values where available.
A common mistake is to see an interim nominal p-value below 0.05 and conclude that the design’s efficacy boundary has been crossed. That conclusion is not justified: for example, \(p_1=0.00939\) is below 0.05 but exceeds the illustrative O’Brien-Fleming interim threshold of 0.00258. The correction is to compare the observed nominal p-value with the threshold for that particular look, or equivalently compare \(z_j\) with \(c_j\).
A second mistake is to treat a nominal threshold as an adjusted p-value for the entire sequential procedure. It is not. It is the ordinary one-look tail area corresponding to one boundary. Sequentially adjusted evidence summaries and overall error calculations answer different questions; the boundary conversion alone does not provide them.
What the P-Value Scale Clarifies—and What It Does Not
The conversion is useful because it places different look-specific standardized cutoffs on a scale familiar to readers of clinical results. It makes the contrast between relatively stable Pocock thresholds and very stringent early O’Brien-Fleming thresholds immediate. It also provides a simple decision check: at look \(j\), compare the one-sided nominal p-value with \(1-\Phi(c_j)\).
The conversion does not change the boundary, make the trial’s repeated looks independent, or turn a nominal p-value into an overall sequential p-value. In particular, nominal thresholds are not incremental allocations of type I error and should not be added. The probability of crossing at least one boundary under \(H_0\) depends on the joint behavior of the statistics across looks and on the complete design. For boundary decisions, keep the comparison local to the applicable look; for claims about the whole design, use its sequential calibration.
Check Your Understanding
Use one-sided upper-tail nominal p-values unless a question says otherwise. The solutions follow the questions.
- A hypothetical look has cutoff \(c_j=2.00\). What is its nominal p-value threshold?
- A Pocock design has the same cutoff at two looks. What follows about its nominal thresholds, and what does not follow about overall type I error?
- At an interim look, an O’Brien-Fleming cutoff is 2.80 and the observed statistic is 2.40. Does the result cross the boundary? Give the observed nominal p-value and approximate threshold.
- Why is it generally incorrect to add nominal p-value thresholds from separate looks to obtain the overall type I error rate?
Solutions. (1) \(1-\Phi(2.00)\approx0.0228\), so the threshold is about 0.0228. (2) Equal cutoffs give equal nominal thresholds because each is transformed by the same function \(1-\Phi\). This does not make the overall type I error equal to that threshold: the trial has repeated opportunities to cross, and the looks are dependent. (3) The observed p-value is \(1-\Phi(2.40)\approx0.00820\); the cutoff threshold is \(1-\Phi(2.80)\approx0.00256\). Since \(2.40<2.80\), equivalently \(0.00820>0.00256\), the boundary is not crossed. (4) The thresholds are marginal tail areas at individual looks, not disjoint probabilities of first crossing. Repeated looks are dependent, so their sum is not generally the probability of crossing at least one boundary.