Information and Sample Size Under Fixed Variance

Why Sample Size Can Track Information

In a group sequential design, an interim look is often described by how much information has accumulated. Under a particular, common model, the information at a look is proportional to the number of observations available. That makes sample size a convenient way to describe information progress—but only when each observation contributes the same amount of information.

This tutorial establishes that relationship for independent observations with a common, fixed variance. The result follows from the model, not from sample size alone: if the variance changes, or observations are not independent, doubling the number of observations need not double the information. The earlier material on statistical information and precision defines information as a measure of precision; here the focus is the precise sample-size relationship for a normal mean.

Model and assumptions: Let $X_1,\ldots,X_n$ be independent observations from $N(\mu,\sigma^2)$, where $\mu$ is the mean parameter of interest and $\sigma^2$ is fixed and known. The observations have the same variance, and each contributes information about the same mean. Under this model, the Fisher information about $\mu$ in $n$ observations is $I_n=n/\sigma^2$.

Here $I_n$ denotes the information in the cumulative sample of size $n$. Its units are the reciprocal of the squared measurement units: if $X_i$ is measured in millimetres, information about $\mu$ is measured in inverse square millimetres. Information is not itself a sample count, although in this model its numerical value is a constant multiple of that count.

Deriving the Proportionality

Independence makes the joint likelihood the product of the individual normal densities. Taking its logarithm gives a sum of $n$ contributions. Differentiating the log-likelihood with respect to $\mu$ shows that each observation contributes the same amount to the curvature of the log-likelihood.

\begin{aligned} \ell(\mu) &= -\frac{n}{2}\log(2\pi\sigma^2) -\frac{1}{2\sigma^2}\sum_{i=1}^{n}(X_i-\mu)^2,\\ \frac{\partial \ell}{\partial \mu} &= \frac{1}{\sigma^2}\sum_{i=1}^{n}(X_i-\mu),\\ \frac{\partial^2 \ell}{\partial \mu^2} &= -\frac{n}{\sigma^2},\\ I_n(\mu) &= -E\!\left[\frac{\partial^2 \ell}{\partial \mu^2}\right] = \frac{n}{\sigma^2}. \end{aligned}

The final equality uses the definition of Fisher information as the expected negative second derivative of the log-likelihood. In this model the second derivative does not depend on the observed values, so taking its expectation leaves it unchanged. The sum of $n$ identical contributions is why information grows linearly with $n$: each additional independent observation adds $1/\sigma^2$ units of information.

The same conclusion can be checked through precision. The sample mean has variance $\operatorname{Var}(\bar{x})=\sigma^2/n$, so its precision, the reciprocal of that variance, is $n/\sigma^2$. Thus Fisher information for $\mu$ agrees with the inverse variance of the sample mean in this model. Its standard error is $\sigma/\sqrt{n}$; doubling the sample size multiplies information by two, but reduces the standard error by a factor of $1/\sqrt{2}$, not by one half.

Result for cumulative looks: If look $k$ has $n_k$ observations, with the same fixed variance $\sigma^2$ at every look, then $I_k=n_k/\sigma^2$. Relative to a final look with sample size $n_K$, the information fraction is $$ \frac{I_k}{I_K}=\frac{n_k/\sigma^2}{n_K/\sigma^2}=\frac{n_k}{n_K}. $$ This ratio applies to cumulative sample sizes under the stated model.

Calculating Information at Several Sample Sizes

The calculation has two inputs: the sample size and the variance on the measurement scale. It is important not to substitute the standard deviation where the formula requires variance. If the standard deviation is given as $\sigma$, first square it to obtain $\sigma^2$.

Worked Example: Information at Three Sample Sizes

Suppose a hypothetical study measures a continuous outcome in millimetres. Assume independent observations with known standard deviation $\sigma=12$ mm. Calculate the information at sample sizes $n=40$, $80$, and $120$.

The fixed variance is $ \sigma^2=(12\text{ mm})^2=144\text{ mm}^2 $. Use $I_n=n/\sigma^2$.

\begin{aligned} I_{40}&=\frac{40}{144}=0.2778\ \text{mm}^{-2},\\ I_{80}&=\frac{80}{144}=0.5556\ \text{mm}^{-2},\\ I_{120}&=\frac{120}{144}=0.8333\ \text{mm}^{-2}. \end{aligned}

Each increase of 40 observations adds $40/144=0.2778\text{ mm}^{-2}$. The information doubles from $n=40$ to $n=80$, just as the sample size doubles; the information at $n=120$ is three times the information at $n=40$. These are cumulative information values, not information for three disjoint groups.

For sequential designs, the observations at one look are ordinarily part of the cumulative data at the next look. The information increment between two looks is therefore the difference in their cumulative information. Under the fixed-variance model, this difference depends only on the number of newly added observations.

Worked Example: Information Increments Across Looks

Consider a hypothetical one-sample design with planned cumulative sample sizes $n_1=30$, $n_2=55$, and $n_3=90$. Assume independent measurements in millimetres with fixed standard deviation $\sigma=10$ mm. Find the cumulative information and the information added between looks.

The variance is $100\text{ mm}^2$. Applying $I_k=n_k/100$ gives:

I_1=\frac{30}{100}=0.3000,\qquad I_2=\frac{55}{100}=0.5500,\qquad I_3=\frac{90}{100}=0.9000 \quad\text{mm}^{-2}.

The first look contains 30 observations. The second look adds $55-30=25$ observations, and the third adds $90-55=35$. Their information increments are $0.3000\text{ mm}^{-2}$, $25/100=0.2500\text{ mm}^{-2}$, and $35/100=0.3500\text{ mm}^{-2}$, respectively.

As a check, the increments sum to the final cumulative information: $0.3000+0.2500+0.3500=0.9000\text{ mm}^{-2}$. The information fractions at the first two looks relative to the third are $0.3000/0.9000=1/3$ and $0.5500/0.9000=0.6111$. They also equal $30/90$ and $55/90$.

Checking the Calculation With Standard Errors

Information can also be computed as the reciprocal of the variance of the estimator when that variance is known under the model. This gives a useful independent check on $I_n=n/\sigma^2$. Do not take the reciprocal of the standard error itself; square the standard error first to obtain the variance.

Worked Example: Verifying Information From the Standard Error

A hypothetical measurement process has fixed standard deviation $\sigma=8$ units and independent observations. Compare information and standard error at $n=25$ and $n=100$.

At $n=25$, the information and standard error are $I_{25}=25/64=0.3906\text{ units}^{-2}$ and $SE(\bar{x})=8/\sqrt{25}=1.6$ units. Checking by inverse variance gives $1/(1.6)^2=1/2.56=0.3906\text{ units}^{-2}$.

At $n=100$, the corresponding values are $I_{100}=100/64=1.5625\text{ units}^{-2}$ and $SE(\bar{x})=8/\sqrt{100}=0.8$ units. The check is $1/(0.8)^2=1/0.64=1.5625\text{ units}^{-2}$.

The sample size and information both increase fourfold, while the standard error is halved. This illustrates why a twofold increase in information does not imply a twofold decrease in standard error: standard error varies with the inverse square root of information.

When the Relationship Is Useful—and When It Can Mislead

Under the assumptions stated above, sample size is an exact proxy for the information fraction: the common variance cancels from the ratio. This is useful when specifying planned cumulative looks or checking whether reported information values are consistent with the assumed model. The cancellation does not mean variance is irrelevant. A larger fixed variance lowers the information at every sample size, even though it leaves the fraction $n_k/n_K$ unchanged.

The relationship can fail if observations do not have equal precision or do not contribute independently. For example, if independent normal observations have different known variances $\sigma_i^2$, their total Fisher information about a common mean is $\sum_i 1/\sigma_i^2$, not generally $n/\sigma^2$ for one common $\sigma^2$. Dependence can also prevent the information from being the sum of identical contributions. In either case, counting observations alone is insufficient to determine information.

In practice, compare the assumptions with the design and the data-generating process. Are outcomes measured on the same scale using comparable procedures? Is the variance plausibly stable across participants and looks? Are observations reasonably independent under the model? Large changes in observed variability, changes in measurement procedures, or dependence between observations are warnings that the simple proportionality may not describe actual precision. Such checks do not prove the model is correct; they help identify when the calculation needs a more suitable model.

A related distinction is between the fixed variance assumed in the information calculation and an estimate of variance calculated from observed data. If the analysis substitutes a changing variance estimate at different looks, its estimated standard errors need not follow the fixed-$\sigma^2$ calculation exactly. The formula $n/\sigma^2$ is the model-based result with a common fixed variance; it should not be presented as an exact description of every empirical standard error.

Common pitfall: “The sample size doubled, so the standard error must have halved.” This confuses information with standard error. With fixed variance, doubling $n$ doubles $I_n$, while $SE(\bar{x})=\sigma/\sqrt{n}$ falls by a factor of $\sqrt{2}$. The standard error is halved only when the sample size is multiplied by four.

The main alternative to calculating information from $n/\sigma^2$ is to calculate it from a known or model-based variance for the estimator, such as $1/\operatorname{Var}(\bar{x})$. Under this normal fixed-variance model, the two calculations agree exactly. The variance-based route can be a useful check, especially when standard errors are already available; the sample-size route makes the equal contribution from each observation explicit. If the assumptions differ, neither route justifies treating the information fraction as the sample-size fraction without checking the variance structure.

Specify the model.
Confirm independent observations, a common target mean, and a fixed common variance $\sigma^2$.

Calculate cumulative information.
For each sample size $n_k$, use $I_k=n_k/\sigma^2$, retaining the outcome units in the information units.

Check increments and ratios.
Subtract adjacent cumulative values for increments, and verify $I_k/I_K=n_k/n_K$ for a fixed common variance.

Check the assumptions.
If precision varies or observations are dependent, do not infer information from sample size alone.

Check Your Understanding

Use the fixed-variance model where appropriate. Solutions follow the questions.

Independent observations have a fixed standard deviation of 5 units. Calculate information at $n=20$ and $n=80$, including units.
For a fixed variance, what happens to the standard error of the sample mean when the sample size is multiplied by nine?
A design has cumulative sample sizes 24 and 60, with common variance $36\text{ mm}^2$. Find the information increment between these looks.
Why is the sample-size information fraction equal to the information fraction under the fixed-variance model?
Give one condition under which $I_n=n/\sigma^2$ should not be applied as written.

Solutions. (1) The variance is $25\text{ units}^2$, so $I_{20}=20/25=0.8000\text{ units}^{-2}$ and $I_{80}=80/25=3.200\text{ units}^{-2}$. (2) Since $SE(\bar{x})=\sigma/\sqrt{n}$, multiplying $n$ by nine divides the standard error by $\sqrt{9}=3$. (3) There are $60-24=36$ added observations, so the increment is $36/36=1.000\text{ mm}^{-2}$. (4) Both cumulative information values contain the same factor $1/\sigma^2$, which cancels in their ratio. (5) For example, unequal observation variances or dependence between observations violates the equal independent contribution assumptions; the information must then be derived under an appropriate model rather than inferred from the sample size alone.