Unequal Stage Sizes and Information-Based Weights

Why Unequal Stage Sizes Change the Weights

When two stages contribute different amounts of information, giving their standardized statistics equal weight generally does not reproduce the evidence in the combined data. A larger stage usually estimates the treatment effect more precisely, so its statistic should contribute more to an information-based combination. The relevant quantity is information, not sample size by itself: sample size is proportional to information only under particular models and allocation schemes.

This tutorial derives the weights for a two-stage normal setting and shows how they work when the stages have unequal sizes. The derivation also gives a useful check: under a common-effect model, the inverse-normal combination with these weights equals the usual pooled normal statistic.

Definition: Let $Z_1$ and $Z_2$ be independent stagewise standard normal test statistics under $H_0$, and let $I_1>0$ and $I_2>0$ be their stage-specific information. The information-based inverse-normal weights are $$ w_1=\sqrt{\frac{I_1}{I_1+I_2}},\qquad w_2=\sqrt{\frac{I_2}{I_1+I_2}}. $$ The combined statistic is $Z_{\mathrm{comb}}=w_1Z_1+w_2Z_2$. These weights satisfy $w_1^2+w_2^2=1$.

Deriving the Weights From Information

Suppose stage $k$ estimates the same normal-model parameter $\theta$ by $\hat{\theta}_k$, with known variance $1/I_k$. The stage estimates are independent, and under the null hypothesis $H_0:\theta=\theta_0$, the stage statistic is $Z_k=(\hat{\theta}_k-\theta_0)\sqrt{I_k}$. Thus each $Z_k$ has a standard normal distribution under $H_0$.

The pooled estimator weights each stage estimate in proportion to its precision, or information. Its variance is the reciprocal of total information. Standardizing that pooled estimator yields the information-based combination directly:

\hat{\theta}_{\mathrm{pool}} =\frac{I_1\hat{\theta}_1+I_2\hat{\theta}_2}{I_1+I_2}, \qquad \operatorname{Var}(\hat{\theta}_{\mathrm{pool}}) =\frac{1}{I_1+I_2}.

\begin{aligned} Z_{\mathrm{pool}} &=(\hat{\theta}_{\mathrm{pool}}-\theta_0)\sqrt{I_1+I_2}\\ &=\frac{I_1(\hat{\theta}_1-\theta_0)+I_2(\hat{\theta}_2-\theta_0)} {\sqrt{I_1+I_2}}\\ &=\sqrt{\frac{I_1}{I_1+I_2}}Z_1+ \sqrt{\frac{I_2}{I_1+I_2}}Z_2. \end{aligned}

So the weights are square roots of the information fractions, not the information fractions themselves. The square roots arise because each stage statistic is already standardized: information affects its contribution on the standard-deviation scale.

For independent standard normal statistics under $H_0$, the combination has mean zero and variance $w_1^2+w_2^2=1$. Therefore $Z_{\mathrm{comb}}\sim N(0,1)$ under the null. For a one-sided alternative $H_a:\theta>\theta_0$, the upper-tail combined p-value is $1-\Phi(Z_{\mathrm{comb}})$, where $\Phi$ is the standard normal cumulative distribution function. This reference distribution requires the stated independence and valid stagewise standard normal statistics.

Information Is Not Always the Sample Size

When each independent observation has variance $\sigma^2$, the sample mean from $n_k$ observations has variance $\sigma^2/n_k$, so $I_k=n_k/\sigma^2$. If the stages share the same variance, the information weights reduce to $w_k=\sqrt{n_k/(n_1+n_2)}$. In that setting, unequal stage sizes translate directly into unequal information.

For a treatment-control difference with $n_k$ participants in each arm and common known outcome variance $\sigma^2$, the variance of the stage difference is $2\sigma^2/n_k$, so its information is $n_k/(2\sigma^2)$. The weights again depend on the square roots of the stage sample-size fractions if the allocation and variance are the same across stages. With different variances, allocation ratios, or precision, calculate information from the variance of the stage effect estimate instead of substituting raw sample size.

Assumptions for the exact normal calculation: The stagewise effect estimates are independent; their normal-model variances are correctly specified; the stage statistics are standardized using those variances; and the weights are fixed independently of the stagewise test results. Under these conditions, the weighted statistic has the stated standard normal null distribution. If a later-stage size is chosen using earlier results, weights calculated from the realized information can be data-dependent. Their validity must then be justified under the design’s error-control conditions; the fixed-weight derivation alone is not sufficient.

That final distinction matters in sample size re-estimation. Unequal sizes specified in advance give fixed unequal weights. By contrast, if the second-stage information is chosen after observing first-stage data, the realized information fraction can depend on the first-stage statistic. The earlier material on data-dependent weights and validity conditions explains why simply inserting that realized fraction does not automatically give a standard normal combined statistic.

Worked Example: Combining Unequal Stage Information

Consider a hypothetical two-stage one-sided test of $H_0:\theta=0$ against $H_a:\theta>0$. Suppose the independent stagewise statistics are $Z_1=1.800$ and $Z_2=1.549$, and stage information is $I_1=150$ and $I_2=250$ information units. Assume the information amounts and weights were fixed independently of the observed test statistics.

The total information is $400$ units, giving $w_1=\sqrt{150/400}=\sqrt{0.375}\approx0.6124$ and $w_2=\sqrt{250/400}=\sqrt{0.625}\approx0.7906$. The weights are normalized because $0.375+0.625=1$.

Substitute the stage statistics into the combination:

Z_{\mathrm{comb}} =\sqrt{0.375}(1.800)+\sqrt{0.625}(1.549) \approx 1.102270+1.224592 =2.326862 \approx 2.327.

The one-sided p-value is $1-\Phi(2.327)\approx0.0100$. Under the assumed model, this is the probability under $H_0$ of obtaining a combined statistic at least as large as the observed value. It is not the probability that $H_0$ is true. At a one-sided significance level of $0.025$, the result is statistically significant because $2.327>1.960$, the standard normal upper-tail critical value for that level.

Examples: Sample-Size Ratios and Pooled Inference

Worked Example: Unequal Sample Sizes With Common Variance

Suppose a hypothetical normal-mean analysis has independent stage samples of $n_1=100$ and $n_2=300$, with the same known variance in both stages. The observed one-sided stage statistics are $Z_1=0.400$ and $Z_2=2.000$. The equal-variance assumption makes information proportional to sample size, so the information fractions are $100/400=0.25$ and $300/400=0.75$.

The weights are $w_1=\sqrt{0.25}=0.5000$ and $w_2=\sqrt{0.75}\approx0.8660$. Therefore,

Z_{\mathrm{comb}}=(0.5000)(0.400)+(0.8660)(2.000) =0.2000+1.7321 =1.9321.

The one-sided p-value is $1-\Phi(1.9321)\approx0.0267$. The larger second stage contributes more because it carries three quarters of the total information. The stages are independent by assumption, and the weights are fixed by their prespecified sample sizes; those conditions support the standard normal reference calculation.

Worked Example: Checking Against a Pooled Treatment Difference

In a hypothetical two-arm normal trial, each arm has 50 participants in stage one and 200 in stage two. The known outcome standard deviation is $\sigma=10$ units in both arms. Suppose the observed treatment-minus-control differences are $2$ units in stage one and $1$ unit in stage two. Assume independent participants across stages and equal allocation within each stage.

For a stage with $n_k$ participants per arm, the variance of the difference is $2\sigma^2/n_k$. Thus stage-one variance is $2(10^2)/50=4$ square units and stage-two variance is $2(10^2)/200=1$ square unit. The corresponding information values are $I_1=1/4=0.25$ and $I_2=1$, in inverse-square units. Hence the information fractions are $0.20$ and $0.80$, and the weights are $w_1=\sqrt{0.20}\approx0.4472$ and $w_2=\sqrt{0.80}\approx0.8944$.

The stage standard errors are $2$ and $1$ units, so $Z_1=2/2=1$ and $Z_2=1/1=1$ for testing a zero treatment difference against a positive difference. The combination is

Z_{\mathrm{comb}}=(0.4472)(1)+(0.8944)(1) \approx1.3416.

As a check, the pooled difference is $(0.25(2)+1(1))/(1.25)=1.200$ units, and its standard error is $\sqrt{1/1.25}\approx0.8944$ units. Therefore the pooled statistic is $1.200/0.8944\approx1.3416$, agreeing with the combination. The upper-tail p-value is approximately $0.0899$, so this example does not reject the null at a one-sided $0.025$ level. The pooled check is valid here because the stages estimate the same effect, have known variances, and are independent.

Diagnostics, Alternatives, and Limitations

Before reporting an information-based combination, check the assumptions and the arithmetic, not just the final statistic. The following checks often reveal errors:

Verify the information scale. Information has inverse-variance units for an effect estimate. If variances differ by stage, using sample-size fractions alone gives the wrong weights.
Check the weights. Each weight must be nonnegative for this one-sided construction, and $w_1^2+w_2^2=1$. The information fraction is squared weight, not weight itself.
Check stage independence. Shared participants, overlapping data, or dependence between stagewise estimates can invalidate the standard normal variance calculation unless the dependence is explicitly accounted for.
Preserve the alternative’s direction. For an upper-tail test, use signed statistics and $1-\Phi(Z_{\mathrm{comb}})$. Reversing the tail changes the question being tested.
Check whether weights are fixed. A weight based on a sample size selected after viewing stage-one results is not automatically a fixed weight. Use only a combination procedure whose error control has been justified for that adaptation.

The main alternative within inverse-normal combination testing is to use other prespecified weights, such as equal weights $1/\sqrt{2}$ for two stages. Equal weights still give a standard normal combination under independent standard normal stage statistics, but they need not match pooled normal inference when information differs. Information-based weights have a specific efficiency rationale under a common-effect normal model: they reproduce the pooled statistic.

Fisher’s combination method is another alternative, based on adding transformed stagewise p-values rather than forming a weighted sum of normal quantiles. It has a different reference distribution and does not have the same direct pooled-statistic identity. The choice between methods depends on the design and the properties needed; the methods are not interchangeable by changing a weight. Information-based inverse-normal weights also have limits: their pooled interpretation relies on compatible stagewise estimands and valid variance estimates, and the fixed-weight null result relies on independence and appropriate weight handling.

Common Mistake: Using Information Fractions as Weights

A tempting but incorrect calculation is to use $I_1/(I_1+I_2)$ and $I_2/(I_1+I_2)$ directly as the coefficients of $Z_1$ and $Z_2$. Those are information fractions, but they do not generally yield a unit-variance combination. For example, fractions $0.25$ and $0.75$ give a weighted sum with null variance $0.25^2+0.75^2=0.625$, not $1$. Its statistic cannot be compared directly with standard normal critical values.

Key takeaway: For independent stagewise standard normal statistics, information-based inverse-normal weights are the square roots of information fractions. They give a standard normal null statistic when the weights are valid and fixed, and under a common normal effect they reproduce the pooled statistic.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Two independent stages have information $I_1=64$ and $I_2=36$. What are the information-based weights?
With the weights from question 1, suppose $Z_1=1.00$ and $Z_2=2.00$. Calculate the combined statistic.
Why is it not generally correct to use sample-size fractions as weights when the stage variances differ?
What additional concern arises if the second-stage sample size, and hence the realized weights, is chosen using the first-stage result?

Solutions. (1) Total information is $100$, so $w_1=\sqrt{64/100}=0.8000$ and $w_2=\sqrt{36/100}=0.6000$. (2) $Z_{\mathrm{comb}}=(0.8000)(1.00)+(0.6000)(2.00)=2.000$. (3) Sample size is proportional to information only when the variance and allocation structure make it so; information must be calculated from the stage estimate’s variance. (4) The weights may then depend on the observed first-stage statistic, so the fixed-weight standard normal argument does not by itself establish type I error control.