Worked Example: Inverse-Normal Re-Estimation

Why the Weights Matter After Re-Estimation

In a two-stage trial, an interim result may determine how many participants are enrolled in stage two. That change can alter the amount of information collected, but it does not automatically make a fixed-design critical value appropriate for the final analysis. An inverse-normal combination test instead transforms each stage’s p-value to a normal score and combines the scores using specified weights.

The key distinction in this example is between planned information used to define the weights and information actually collected after the interim decision. When the sample-size rule depends on interim data, using the realized information to change the weights can also make the weights depend on those data. The resulting statistic is not automatically standard normal. Here, the weights are fixed before outcomes are observed, while the second-stage sample size follows a prespecified interim rule.

Model, Assumptions, and Weight Choice

Suppose each stage estimates the same treatment effect \(\theta\), measured in millimeters of mercury (mmHg). For clarity, take the stagewise estimators to be normal with known standard errors: \(\hat{\theta}_j\sim N(\theta,1/I_j)\), where \(I_j\) is the information in stage \(j\). Thus \(Z_j=\hat{\theta}_j\sqrt{I_j}\) has variance 1 and mean \(\theta\sqrt{I_j}\). The one-sided stagewise p-value is \(p_j=1-\Phi(Z_j)\).

The validity argument requires independent stagewise data, a correctly specified normal test (or a stagewise procedure that gives valid p-values), and an interim rule that uses only information available before stage two. In particular, no stage-two participant or outcome may be reused in stage one. Conditional on the interim data and the chosen stage-two size, the stage-two test must still give a valid one-sided p-value under the null. At the boundary \(\theta=0\), this p-value is uniform conditional on the interim data in the continuous normal model.

A common information-based choice uses the original planned information values \(I_1^*\) and \(I_2^*\):

These weights are nonnegative and satisfy \(w_1^2+w_2^2=1\). For an adaptation that changes the second-stage size, the procedure below keeps these original weights fixed. It does not replace \(I_2^*\) with the outcome-dependent realized information when calculating the weights. A procedure using other weight rules requires its own calibration argument.

Why the Fixed-Weight Combination Is Calibrated

The justification can be seen by conditioning on the interim data. Under the boundary null \(\theta=0\), \(Z_1\sim N(0,1)\). Given the stage-one data and the selected stage-two size, the independent stage-two statistic has \(Z_2\sim N(0,1)\). Because this conditional distribution does not depend on the interim result, \(Z_2\) is independent of \(Z_1\) under the boundary null. With fixed weights, the sum is therefore normal with mean zero and variance one:

Thus the rule that selects the second-stage size does not, by itself, alter the null distribution of this combination statistic. The argument relies on the stage-two p-value remaining conditionally valid and on weights that do not respond to the interim result. For the composite null \(H_0:\theta\leq0\), the stagewise one-sided p-values are super-uniform under the null, and the conditional-validity argument gives Type I error control at the boundary and no larger rejection probability under the null.

This is a short but important derivation: the standard normal reference distribution follows from the independent, standard normal stagewise scores and the sum-of-squared-weights condition. It is not a consequence of merely having two normally distributed estimates. Selection, reuse of data, invalid stagewise p-values, or weights that vary with interim outcomes can break the argument.

Realized-Information Weights: A Calibration Warning

Information-based weights are appealing because, in a fixed design, they reflect how much information each stage contributes. But in an adaptive design, the realized second-stage information may be a function of \(Z_1\). Recomputing both weights from that realized information makes the weights functions of the interim data. The fixed-weight derivation above then no longer applies.

Checks, Alternatives, and Limitations

Before reporting an inverse-normal result, verify that the implemented analysis matches the prespecified design. In particular, check that the rule’s branch and stage-two size are correct, that both stagewise p-values correspond to the intended one-sided hypothesis, and that the weight values were fixed using the stated reference information. Also check that the second-stage data are independent of stage one and that any changes to the endpoint, analysis population, or test preserve conditional validity.

A naive pooled test is not a substitute: its usual fixed-design reference distribution may fail when sample size is selected using interim outcomes. Likewise, using a fixed-design critical value after re-estimation does not automatically control the overall Type I error. The inverse-normal method combines stagewise evidence under its stated conditions; it does not make every adaptive analysis valid. A conditional-error recalibration is a different approach when the planned final testing rule must be modified, and other combination functions use different ways of combining p-values.

Fixed weights also have a cost. They need not reflect the realized information split, so they may not be the most powerful choice for every adaptation rule or alternative effect. The method’s validity does not guarantee optimal power, and the weights and sample-size rule should be specified and justified before the relevant outcomes are observed. If the analysis uses estimated variance, discrete outcomes, or a test with ties, stagewise p-value validity and calibration require particular care.

Common Mistakes

• Recalculating weights from the realized second-stage size without justification. If that size depends on interim data, the resulting weights are data-dependent; the fixed-weight standard normal argument no longer applies automatically.
• Combining the p-values directly as if they were normal scores. The inverse-normal rule first uses \(Z_j=\Phi^{-1}(1-p_j)\), then combines those scores.
• Using a two-sided p-value in a one-sided formula without a prespecified direction convention. The stagewise p-values here are upper-tail p-values for evidence that \(\theta>0\).
• Assuming that independent stagewise data alone guarantee validity. The weights, stagewise tests, and interim rule must also satisfy the conditions used in the calibration argument.