Why the Timing of Re-Estimation Matters
A sample-size update can occur before or after an interim efficacy result is examined. The difference is important: an update based only on a nuisance parameter, such as outcome variability, does not directly select participants according to an observed treatment effect. An update made after an efficacy review can depend on a noisy estimate of that effect, so the ordinary final analysis may no longer have its planned null distribution.
“Before an efficacy look” does not mean before any trial outcomes have been collected. It means that the re-estimation decision is completed before the treatment-effect comparison is reviewed by the people making or directing that decision. For example, a separate statistician might use accumulating outcomes to estimate a pooled variance, report only the revised sample size, and withhold treatment-effect summaries. The data may be blinded or operationally unblinded for the nuisance calculation; what matters statistically is what information drives the update and what the final test does with the data.
This is a timing distinction, not a guarantee of validity. The protocol still needs to specify the re-estimation rule, the information available to those applying it, any sample-size floor and cap, and a final inferential procedure appropriate to the adaptation. Earlier material on maintaining a fixed final test after re-estimation is relevant here: a fixed test can be retained only when its validity under the actual adaptation has been justified.
What Must Be Specified
A useful pre-efficacy re-estimation plan sets out the decision before unblinded efficacy results are available. It should define the target nuisance parameter and estimator, the time and data cut used to estimate it, the formula translating the estimate into a sample size, and the rounding and bounds. It should also identify who can see treatment assignments or efficacy summaries, what information is released to the trial team, and how the final analysis handles the accumulated data.
For a simple two-arm comparison of independent normally distributed outcomes with a common variance \(\sigma^2\), equal allocation, a two-sided test at level \(\alpha\), and target power \(1-\beta\) for a clinically relevant difference \(\delta\), a planning approximation for the number per arm is
This approximation assumes independent observations, a common within-arm variance, a fixed target difference, and the usual normal approximation for the test statistic. Replacing \(\sigma\) by an interim estimate gives a proposed revised sample size; it does not establish that a particular final test retains its nominal type I error. The formula also does not account for every design feature, such as clustering, repeated measures, or an adaptive group-sequential boundary.
Worked Example: Updating the Planned Size From a Variance Estimate
Suppose a hypothetical trial compares mean change in a continuous clinical score between two equally allocated arms. The target difference is \(\delta=5\) score points, the planned two-sided level is \(\alpha=0.05\), and target power is \(90\%\). The original planning standard deviation is \(12\) points. Before any interim efficacy review, an independent statistician estimates a pooled within-arm standard deviation of \(15\) points from the available outcomes. Assume the common-variance normal model and the planning approximation above are appropriate.
Using \(z_{0.975}=1.960\) and \(z_{0.90}=1.282\), the updated number per arm is
Rounding upward gives \(190\) participants per arm, or \(380\) in total. For comparison, the original estimate of \(12\) points gives
which rounds upward to \(122\) per arm. The larger variance estimate therefore increases the planned size under this formula. This calculation assumes that the variance estimate is suitable for the intended population and outcome, that the target difference and testing plan have not changed, and that \(190\) per arm is within the pre-specified cap. It is a planning calculation, not a demonstration of type I error control for the final analysis.
Why Nuisance-Only Timing Can Help
The central statistical advantage is that the sample-size decision need not be selected for a promising or disappointing interim treatment effect. In a well-run nuisance-only process, the efficacy comparison is not used to decide whether to enroll more participants. This can avoid one important source of data-dependent selection. However, “the treatment effect was not shown to the decision-maker” is an operational fact; type I error control is a statistical property of the complete design and analysis.
A general justification uses conditioning. Let \(A\) denote all information that determines the sample-size update, and let \(R\) denote rejection of \(H_0\) at the final analysis. If the final procedure satisfies \(P(R\mid A,H_0)\leq\alpha\) for every permitted value of \(A\), then
This is the law of total probability applied to the conditional rejection probability. The key requirement is the conditional bound, not merely that the update was called “nuisance-only.” Whether it holds depends on the estimator, the data reused in the final test, the sample-size rule, and the specified analysis. In particular, independence properties that hold for a normal model do not automatically carry over to other outcome distributions or analysis procedures.
The argument also explains why the operational firewall matters but is not enough on its own. If the final procedure has not been shown to satisfy the needed conditional bound, restricting access to the interim treatment effect does not prove that the unconditional rejection probability is at most \(\alpha\). The design must use a final test whose validity under the actual update rule is established.
Worked Example: Checking a Conditional Error Claim
Consider a hypothetical design with two possible nuisance-information outcomes, \(A=a_1\) or \(A=a_2\), which lead to different sample sizes. Suppose the planned final procedure has conditional rejection probabilities \(0.025\) and \(0.024\), respectively, under \(H_0\). If \(P(A=a_1\mid H_0)=0.60\) and \(P(A=a_2\mid H_0)=0.40\), then
The overall type I error is \(0.0246\), below the planned \(\alpha=0.025\). The conclusion follows because the conditional rejection probability is at most \(0.025\) in both cases. If only the weighted average had been reported, that would establish the overall error for these stated probabilities, but it would not establish the stronger conditional guarantee. In an actual trial, the conditional probabilities must be justified for the specified test and adaptation, not assumed from this illustration.
Contrast With Re-Estimation After an Efficacy Review
After an unblinded interim efficacy analysis, the update may depend on an observed treatment difference, a test statistic, or conditional power under a chosen effect assumption. Under \(H_0\), these interim quantities still vary by chance. If the rule increases sample size when the interim result looks favorable, the final analysis is applied to a design that was selected partly by that favorable noise. An ordinary fixed-sample test may then have a different rejection probability from its nominal level.
Worked Example: A Sample-Size Rule Triggered by an Efficacy Statistic
Suppose a hypothetical trial plans a one-sided interim efficacy statistic \(Z\) with \(Z\sim N(0,1)\) under \(H_0\). The rule is to increase the maximum sample size if \(Z>1.50\). Under the null, the probability of triggering that increase is
Thus, even when there is no treatment effect, about \(6.68\%\) of trials would trigger the increase solely because of random interim variation under this model. If the final analysis then treats the resulting data as though the sample size had been fixed independently of the interim result, its nominal error rate is not justified by the original fixed-sample calculation. This example does not calculate the resulting error rate; that requires the full adaptation and final-test rules. It shows why efficacy-driven updates need an analysis designed for that dependence.
| Feature | Before efficacy review | After unblinded efficacy review |
|---|---|---|
| Information driving the update | Specified nuisance information only, if the rule is genuinely nuisance-only | May include interim treatment-effect evidence |
| Main concern | Estimator bias, information leakage, and validity of the final test after data-dependent sizing | Selection on interim efficacy evidence and dependence between the update and final statistic |
| Operational safeguard | Separate access to nuisance summaries; do not release the efficacy comparison before the decision | Pre-specified adaptive inferential method and controlled access to interim results |
| What timing alone establishes | That the update preceded efficacy review, not that type I error is controlled | Nothing about validity without an appropriate analysis |
Diagnostics, Limitations, and Common Mistakes
The process is not behaving as intended if the revised sample size changes in response to an efficacy summary, the decision-makers can reconstruct that summary from the information they receive, or the data cut and estimator differ from the protocol. Statistical checks should also examine whether the estimated nuisance parameter is plausible, whether the calculation is sensitive to a few unusual observations, and whether the chosen cap leaves the intended power attainable. For a variance-based rule, a marked departure from the assumed outcome distribution or strong evidence of unequal variances is a reason to question the planning model, not a reason to treat the recalculated size as automatically adequate.
A common mistake is: “The update happened before the efficacy look, so the original test is automatically valid.” The correction is that timing can prevent the update from using the treatment-effect result, but the final test must still be valid under the specific adaptation and reuse of data. Another mistake is to call a procedure blinded when someone making the update has access to arm-specific outcome summaries. State precisely who sees which data and which summaries influence the rule.
Compared with an efficacy-driven update, nuisance-only re-estimation can make the decision rule easier to separate from treatment-effect evidence and easier to protect with an operational firewall. Its limitations remain substantial: an interim nuisance estimate can be imprecise or unrepresentative, a cap can prevent the target power from being reached, and an apparently neutral statistic can still leak efficacy information. Conversely, post-efficacy adaptation is not inherently invalid; it requires an inferential method that accounts for the interim result and the sample-size change. Neither approach should be selected on timing alone.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Does a re-estimation performed before the efficacy review necessarily use no trial outcomes? Explain.
- In the normal two-arm example, what is the revised total sample size when the updated standard deviation is \(15\) points and the rounded per-arm target is \(190\)?
- Why does a conditional rejection probability no greater than \(\alpha\) for every permitted adaptation outcome imply an overall rejection probability no greater than \(\alpha\)?
- Under the null, what is the probability that a standard normal interim statistic exceeds \(1.50\)? What does this say about an update triggered by that threshold?
- Name two safeguards or design specifications needed in a nuisance-only re-estimation plan.
Solutions: (1) No. Accumulating outcomes can be used to estimate a nuisance parameter; the defining feature is that the efficacy evidence is not reviewed to drive the update. (2) \(2(190)=380\) participants in total. (3) Averaging the conditional rejection probabilities over the adaptation information, by the law of total probability, preserves the upper bound \(\alpha\). (4) \(1-\Phi(1.50)\approx0.0668\), or \(6.68\%\); a null trial can trigger the increase through random interim variation, so an efficacy-dependent update requires an analysis that accounts for it. (5) Examples include pre-specifying the estimator and data cut, defining sample-size floors and caps, restricting access to efficacy summaries, and specifying a final test valid under the adaptation.