Why Scale a Selection Cutoff by Information?
An interim selection rule needs a clear answer to a practical question: how much evidence must an experimental arm show to continue? A cutoff stated on the effect scale can have a constant interpretation across interim looks, but its corresponding standardized-score cutoff changes as information accumulates. Conversely, keeping the standardized-score cutoff fixed while information changes alters the effect size required for retention.
This tutorial develops the conversion between these scales. It focuses on selection, not on establishing a confirmatory testing boundary. An information-based cutoff can make the effect threshold explicit, but by itself it does not guarantee a particular familywise error rate or determine the probability an arm will continue.
For a simple derivation, suppose the estimated treatment effect for arm \(k\) at look \(j\) is \(\hat\theta_{kj}\), with standard error \(1/\sqrt{I_{kj}}\). The standardized score is then \(Z_{kj}=\hat\theta_{kj}\sqrt{I_{kj}}\). This is an exact variance relationship in a normal model with known variance; in other settings it can be an approximation when the standard error is well estimated. A positive score is taken to favor the experimental arm.
Deriving the Information-Based Threshold
Suppose the selection policy requires the estimated effect to be at least \(\delta\), where \(\delta>0\) is a prespecified minimum effect, measured in the same units as \(\hat\theta_{kj}\). The arm qualifies when \(\hat\theta_{kj}\geq\delta\). Since \(Z_{kj}=\hat\theta_{kj}\sqrt{I_{kj}}\), multiplying both sides of that inequality by the positive quantity \(\sqrt{I_{kj}}\) gives an equivalent rule on the standardized-score scale.
Thus the score cutoff is \(c_{kj}=\delta\sqrt{t_{kj}I_k^*}\). The threshold increases with information: with more precise data, a larger standardized score corresponds to the same minimum effect on the original scale. The effect-scale threshold remains \(\delta\); the standardized threshold changes because its standard-error unit changes. If information is multiplied by a factor \(r\), the score cutoff is multiplied by \(\sqrt r\), not by \(r\).
Worked Example: Calculating a Planned Information-Based Cutoff
Suppose the planned final information for an arm is \(I_k^*=144\), in inverse squared effect units, and the selection policy requires an estimated effect of at least \(\delta=0.25\) effect units. At an interim, planned to occur at information fraction \(t_{kj}=0.50\), calculate the score cutoff. Suppose the observed score is \(2.20\).
First calculate the planned interim information: \(I_{kj}=0.50(144)=72\). The cutoff is
Because \(2.20\geq2.12\), the arm meets this selection rule. The equivalent effect estimate is \(\hat\theta_{kj}=Z_{kj}/\sqrt{I_{kj}}=2.20/\sqrt{72}=0.2593\) effect units, which is above \(0.25\). The score comparison and effect-scale comparison therefore agree.
Actual Information Versus the Scheduled Look
In practice, the information available at an interim can differ from the plan. Recruitment, outcome availability, event rates, or variance estimates may make the accrued information fraction smaller or larger than expected. If the selection policy is defined by a minimum effect, the cutoff should be computed from that actual information. A threshold tied only to the scheduled visit does not automatically preserve the same effect-scale requirement.
Worked Example: More Information Than Planned
Continue with \(I_k^*=144\) and \(\delta=0.25\). A look scheduled at information fraction \(0.60\) instead has actual information fraction \(0.70\). The observed interim score is \(Z_{kj}=2.31\). Determine whether the arm qualifies under the actual-information rule, and compare the cutoff with the scheduled one.
Actual information is \(I_{kj}=0.70(144)=100.8\). Therefore the actual-information cutoff is
The observed score \(2.31\) is below \(2.51\), so the arm does not qualify under this rule. Its estimated effect is \(2.31/\sqrt{100.8}=0.2301\) effect units, also below \(0.25\).
For comparison, the cutoff calculated from the scheduled fraction alone is \(0.25\sqrt{0.60(144)}=0.25\sqrt{86.4}=2.3238\approx2.32\). The observed score is below that cutoff too, so this particular comparison gives the same decision under both rules. The cutoffs are nevertheless different, and cases closer to the boundary can produce different decisions.
Information-Based and Fixed Scheduled Cutoffs
A fixed scheduled cutoff is set using the information expected at a particular calendar visit and then applied without adjustment if actual information differs. An information-based cutoff is recalculated from the information actually accrued. The distinction is not simply that one rule is “stricter”: it is whether the cutoff continues to represent the same effect-scale requirement when information changes.
| Rule | How the cutoff is obtained | Effect-scale implication if actual information changes |
|---|---|---|
| Information-based | Use \(c_{kj}=\delta\sqrt{I_{kj}}\) with actual interim information | Preserves the requirement \(\hat\theta_{kj}\geq\delta\), under the stated model |
| Fixed scheduled | Use the cutoff calculated at the planned information for the calendar look | The implied minimum effect can be higher or lower than \(\delta\) if actual information differs |
Worked Example: A Different Decision Under a Fixed Cutoff
Suppose \(I_k^*=144\), \(\delta=0.25\), and a calendar visit is scheduled at \(t=0.50\), but actual information is \(t=0.70\). The observed score is \(2.40\). Compare applying the actual-information rule with carrying forward the planned calendar cutoff unchanged.
The actual-information cutoff is \(0.25\sqrt{0.70(144)}=0.25\sqrt{100.8}=2.50998\approx2.51\). Since \(2.40<2.51\), the arm does not qualify under the information-based rule. On the effect scale, the estimate is \(2.40/\sqrt{100.8}=0.2390\), below \(0.25\).
The fixed scheduled cutoff is \(0.25\sqrt{0.50(144)}=0.25\sqrt{72}=2.1213\approx2.12\). Since \(2.40\geq2.12\), the arm qualifies under this fixed-cutoff rule. At the actual information, however, that score cutoff corresponds to an effect of \(2.1213/\sqrt{100.8}=0.2113\) effect units, not \(0.25\). The different decision follows from the different policies: the fixed cutoff no longer enforces the original minimum-effect requirement when the information fraction has changed.
Assumptions, Diagnostics, and Limits
The conversion is useful only if its inputs and interpretation are appropriate. At minimum, the analyst should specify the direction of benefit, the effect scale, the minimum effect \(\delta\), the information definition, and whether the cutoff uses actual or planned information. For a multi-arm trial, information may differ by arm; the relevant \(I_{kj}\) should be used for each arm rather than assuming all arms have identical precision.
The relation \(Z_{kj}=\hat\theta_{kj}\sqrt{I_{kj}}\) requires the standard error to equal \(1/\sqrt{I_{kj}}\), or to be adequately represented by that form. With estimated variances or other score constructions, check that the chosen information measure matches the variance of the score. If information is estimated unreliably, the threshold can inherit that uncertainty. An estimated information fraction slightly above one should prompt a check of the planned information and its calculation; if additional information is genuinely accrued, the formula can be evaluated with the actual information, but the design must state how such overshoot is handled.
Diagnostics should include verifying that information is positive and calculated on the prespecified scale, recomputing the threshold independently, and checking that the score and effect-scale decisions agree. A disagreement between \(Z_{kj}\geq\delta\sqrt{I_{kj}}\) and \(\hat\theta_{kj}\geq\delta\) signals an arithmetic, sign, or scale inconsistency. Also check that the cutoff direction matches the definition of benefit: if smaller effects are better, the score direction and inequality must be reversed consistently.
This rule is not interchangeable with a conditional-power rule. A minimum-effect selection criterion asks whether the interim estimate reaches a specified effect threshold. Conditional power instead concerns the probability of meeting a future criterion given interim data and assumptions about future outcomes. Nor does this threshold alone provide an error-control guarantee for later testing after data-dependent arm selection. Its purpose is to make the selection requirement explicit and responsive to information, not to replace design-level calibration.
Common Mistake
Wrong reasoning: “The look is scheduled at 60% information, so its cutoff must stay at the 60% value, even though actual information is 70%.” This treats calendar position as if it uniquely determined precision. Correction: first identify the policy. If the policy requires an estimated effect of at least \(\delta\), calculate the score cutoff from actual information, \(c_{kj}=\delta\sqrt{I_{kj}}\). A fixed calendar cutoff is a different rule and can imply a different minimum effect when actual information departs from plan.
A second error is to multiply the effect threshold by the information fraction without taking a square root. The score is measured in standard-error units, and standard errors scale as \(1/\sqrt{I}\); therefore the standardized cutoff scales as \(\sqrt I\), as the derivation shows.
Check Your Understanding
Use the model and notation defined above. Answers are given after the questions so that each calculation can be checked.
- With \(I_k^*=100\), \(\delta=0.20\), and actual \(t_{kj}=0.64\), calculate the information-based score cutoff.
- For the setting in Question 1, an observed score is \(1.50\). Does the arm qualify? Give the corresponding effect estimate.
- A visit is scheduled at \(t=0.40\), but actual information is \(t=0.60\). Explain why applying the scheduled cutoff unchanged may alter the effect threshold enforced.
- If information is multiplied by four while \(\delta\) stays fixed, by what factor does the standardized-score cutoff change? Explain briefly.
Solutions. (1) Actual information is \(0.64(100)=64\), so \(c=0.20\sqrt{64}=1.60\). (2) The score \(1.50\) is below \(1.60\), so the arm does not qualify. Its estimated effect is \(1.50/\sqrt{64}=0.1875\) effect units, below \(0.20\). (3) The scheduled cutoff is based on less information than was actually accrued. When applied at the higher actual information, that unchanged score cutoff corresponds to a smaller effect threshold than the one used to calculate it; it therefore need not enforce the intended \(\hat\theta_{kj}\geq\delta\) rule. (4) The cutoff is proportional to \(\sqrt I\), so multiplying information by four multiplies the cutoff by \(\sqrt4=2\).